Sep 14 02:49:01.162560 ip-10-0-133-188 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 14 02:49:01.162697 ip-10-0-133-188 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 14 02:49:01.162790 ip-10-0-133-188 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 14 02:49:01.163464 ip-10-0-133-188 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 14 02:49:11.323543 ip-10-0-133-188 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 14 02:49:11.323564 ip-10-0-133-188 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 908eef59e0dc46338dfe7ff276069f1f --
Sep 14 02:50:53.592797 ip-10-0-133-188 systemd[1]: Starting Kubernetes Kubelet...
Sep 14 02:50:53.986002 ip-10-0-133-188 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 02:50:53.986002 ip-10-0-133-188 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 14 02:50:53.986002 ip-10-0-133-188 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 02:50:53.986002 ip-10-0-133-188 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 14 02:50:53.986002 ip-10-0-133-188 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 02:50:53.987125 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.987058    2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 14 02:50:53.992796 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992782    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:50:53.992796 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992796    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992800    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992803    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992806    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992809    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992812    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992815    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992817    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992820    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992822    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992825    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992828    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992831    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992834    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992836    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992839    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992841    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992844    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992848    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:50:53.992860 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992851    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992854    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992857    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992860    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992863    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992866    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992869    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992871    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992874    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992877    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992879    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992881    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992884    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992887    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992889    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992891    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992894    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992896    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992899    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:50:53.993303 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992902    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992905    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992909    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992912    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992915    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992918    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992921    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992923    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992926    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992928    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992930    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992933    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992935    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992938    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992941    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992943    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992946    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992948    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992951    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992953    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:50:53.993776 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992956    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992958    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992961    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992963    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992965    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992968    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992972    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992974    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992976    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992979    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992981    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992984    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992986    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992989    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992991    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992993    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992996    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.992998    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993001    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993004    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:50:53.994261 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993006    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993010    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993012    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993015    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993017    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993020    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993022    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993026    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993030    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993034    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993037    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993039    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993042    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993044    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993046    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993050    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993052    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993055    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993057    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:50:53.994748 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993060    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993745    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993752    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993755    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993758    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993761    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993763    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993766    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993768    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993771    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993773    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993776    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993779    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993782    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993785    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993787    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993790    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993792    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993795    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993798    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:50:53.995201 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993801    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993804    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993807    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993809    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993812    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993814    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993817    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993819    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993821    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993824    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993826    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993829    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993831    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993833    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993836    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993838    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993840    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993843    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993845    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993848    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:50:53.995717 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993850    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993852    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993855    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993858    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993861    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993863    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993866    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993868    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993871    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993873    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993875    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993878    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993881    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993883    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993886    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993888    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993891    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993893    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993895    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993898    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:50:53.996200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993900    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993903    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993905    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993908    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993910    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993913    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993915    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993917    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993920    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993922    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993925    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993927    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993929    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993932    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993934    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993936    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993939    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993941    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993944    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993946    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:50:53.996711 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993949    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993951    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993955    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993960    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993965    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993969    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993972    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993975    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993978    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993981    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993984    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993987    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993990    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993992    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993995    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.993997    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994000    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994003    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:50:53.997189 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994007    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994010    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994380    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994385    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994388    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994392    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994395    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994398    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994401    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994405    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994408    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994410    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994413    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994416    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994419    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994421    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994424    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994426    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994430    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994432    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:50:53.997634 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994435    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994438    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994440    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994443    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994446    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994448    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994451    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994453    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994456    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994458    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994461    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994464    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994467    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994471    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994473    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994476    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994478    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994481    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994483    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:50:53.998120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994486    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994489    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994492    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994494    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994496    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994499    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994502    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994504    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994506    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994509    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994530    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994535    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994539    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994543    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994546    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994549    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994551    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994554    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994556    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994559    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:50:53.998606 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994562    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994564    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994567    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994569    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994572    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994574    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994577    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994581    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994585    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994588    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994590    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994593    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994595    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994598    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994600    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994602    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994606    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994608    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994611    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:50:53.999089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994614    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994616    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994619    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994621    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994624    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994626    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994629    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994631    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994634    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994636    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994639    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994641    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994644    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994646    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994649    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994651    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994654    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994656    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994659    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994661    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:50:53.999601 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994663    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994666    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.994668    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994741    2574 flags.go:64] FLAG: --address="0.0.0.0"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994776    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994782    2574 flags.go:64] FLAG: --anonymous-auth="true"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994785    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994790    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994793    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994797    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994800    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994803    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994806    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994812    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994815    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994818    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994821    2574 flags.go:64] FLAG: --cgroup-root=""
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994824    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994827    2574 flags.go:64] FLAG: --client-ca-file=""
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994829    2574 flags.go:64] FLAG: --cloud-provider="external"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994832    2574 flags.go:64] FLAG: --cluster-dns="[]"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994836    2574 flags.go:64] FLAG: --cluster-domain=""
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994838    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994841    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 14 02:50:54.000120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994844    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994848    2574 flags.go:64] FLAG: --container-log-max-files="5"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994851    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994854    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994857    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994860    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994863    2574 flags.go:64] FLAG: --contention-profiling="false"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994866    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994868    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994871    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994874    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994877    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994880    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994883    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994886    2574 flags.go:64] FLAG: --enable-load-reader="false"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994888    2574 flags.go:64] FLAG: --enable-server="true"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994891    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994895    2574 flags.go:64] FLAG: --event-burst="100"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994898    2574 flags.go:64] FLAG: --event-qps="50"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994900    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994903    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994908    2574 flags.go:64] FLAG: --eviction-hard=""
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994913    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994916    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994920    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 14 02:50:54.000721 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994923    2574 flags.go:64] FLAG: --eviction-soft=""
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994926    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994928    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994931    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994934    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994936    2574 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994939    2574 flags.go:64] FLAG: --fail-swap-on="true"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994942    2574 flags.go:64] FLAG: --feature-gates=""
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994945    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994948    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994951    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994953    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994956    2574 flags.go:64] FLAG: --healthz-port="10248"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994960    2574 flags.go:64] FLAG: --help="false"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994962    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994965    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994968    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994971    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994974    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994977    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994980    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994983    2574 flags.go:64] FLAG: --image-service-endpoint=""
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994985    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 14 02:50:54.001355 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994988    2574 flags.go:64] FLAG: --kube-api-burst="100"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994990    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994993    2574 flags.go:64] FLAG: --kube-api-qps="50"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994996    2574 flags.go:64] FLAG: --kube-reserved=""
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.994999    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995005    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995008    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995011    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995015    2574 flags.go:64] FLAG: --lock-file=""
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995018    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995020    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995023    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995028    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995031    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995034    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995036    2574 flags.go:64] FLAG: --logging-format="text"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995040    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995043    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995045    2574 flags.go:64] FLAG: --manifest-url=""
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995048    2574 flags.go:64] FLAG: --manifest-url-header=""
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995052    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995055    2574 flags.go:64] FLAG: --max-open-files="1000000"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995059    2574 flags.go:64] FLAG: --max-pods="110"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995062    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995064    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 14 02:50:54.001949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995067    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995070    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995073    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995075    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995078    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995085    2574 flags.go:64] FLAG: --node-status-max-images="50"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995088    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995091    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995093    2574 flags.go:64] FLAG: --pod-cidr=""
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995096    2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995101    2574 flags.go:64] FLAG: --pod-manifest-path=""
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995104    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995108    2574 flags.go:64] FLAG: --pods-per-core="0"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995111    2574 flags.go:64] FLAG: --port="10250"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995113    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995116    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b0fcdfce940ff466"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995121    2574 flags.go:64] FLAG: --qos-reserved=""
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995124    2574 flags.go:64] FLAG: --read-only-port="10255"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995126    2574 flags.go:64] FLAG: --register-node="true"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995129    2574 flags.go:64] FLAG: --register-with-taints=""
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995132    2574 flags.go:64] FLAG: --registry-burst="10"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995135    2574 flags.go:64] FLAG: --registry-qps="5"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995138    2574 flags.go:64] FLAG: --reserved-cpus=""
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995140    2574 flags.go:64] FLAG: --reserved-memory=""
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995144    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 14 02:50:54.002849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995146    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995149    2574 flags.go:64] FLAG: --rotate-certificates="false"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995152    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995154    2574 flags.go:64] FLAG: --runonce="false"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995157    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995160    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995163    2574 flags.go:64] FLAG: --seccomp-default="false"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995165    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995168    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995171    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995174    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995177    2574 flags.go:64] FLAG: --storage-driver-password="root"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995179    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995182    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995184    2574 flags.go:64] FLAG: --storage-driver-user="root"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995187    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995190    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995193    2574 flags.go:64] FLAG: --system-cgroups=""
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995195    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995202    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995204    2574 flags.go:64] FLAG: --tls-cert-file=""
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995207    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995211    2574 flags.go:64] FLAG: --tls-min-version=""
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995216    2574 flags.go:64] FLAG: --tls-private-key-file=""
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995219    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 14 02:50:54.003920 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995222    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995224    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995227    2574 flags.go:64] FLAG: --v="2"
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995231    2574 flags.go:64] FLAG: --version="false"
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995235    2574 flags.go:64] FLAG: --vmodule=""
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995238    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.995241    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995355    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995360    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995364    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995367    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995371    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995374    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995377    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995379    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995382    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995385    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995388    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995391    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995394    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995396    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:50:54.004727 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995399    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995402    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995404    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995406    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995409    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995412    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995415    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995418    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995422    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995426    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995429    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995431    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995434    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995436    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995439    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995441    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995444    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995446    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995449    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995451    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:50:54.005262 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995453    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995456    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995458    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995461    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995463    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995466    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995468    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995471    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995473    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995476    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995479    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995481    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995483    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995486    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995489    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995491    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995493    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995496    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995499    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995502    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:50:54.006200 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995504    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995526    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995531    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995534    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995538    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995540    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995544    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995547    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995550    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995552    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995555    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995558    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995571    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995574    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995577    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995580    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995583    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995585    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995588    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:50:54.006951 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995590    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995593    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995596    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995598    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995601    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995603    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995606    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995609    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995611    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995614    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995616    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995619    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995622    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995624    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995628    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995638    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995641    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995644    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995647    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995649    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:50:54.007543 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995652    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:50:54.008374 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995654    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:50:54.008374 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995657    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:50:54.008374 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995659    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:50:54.008374 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995662    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:50:54.008374 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:53.995664    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:50:54.008374 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:53.996321    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 02:50:54.008374 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.004292    2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 14 02:50:54.008374 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.004309    2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 14 02:50:54.008374 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.005221    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 14 02:50:54.008374 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.005234    2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 14 02:50:54.008374 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005316    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:50:54.008374 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005327    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:50:54.008374 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005332    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:50:54.008374 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005337    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005342    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005346    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005350    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005355    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005359    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005363    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005367    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005371    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005375    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005380    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005384    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005388    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005392    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005397    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005401    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005405    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005409    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005413    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:50:54.008885 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005417    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005421    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005425    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005429    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005433    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005437    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005441    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005447    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005451    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005455    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005459    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005463    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005467    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005471    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005475    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005479    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005484    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005487    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005492    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:50:54.009349 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005498    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005503    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005508    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005533    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005537    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005541    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005545    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005549    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005553    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005557    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005561    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005565    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005569    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005573    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005578    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005582    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005586    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005590    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005595    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005598    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:50:54.009961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005603    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005608    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005613    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005617    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005621    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005625    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005629    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005633    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005637    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005641    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005645    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005649    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005653    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005657    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005661    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005665    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005670    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005674    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005678    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005682    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:50:54.010605 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005686    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005690    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005694    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005698    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005702    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005706    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005710    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005714    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005718    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005722    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005726    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005730    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005735    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005738    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005745    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005752    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005757    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:50:54.011141 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005760    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:50:54.011604 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.005768    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 02:50:54.011604 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005938    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:50:54.011604 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005947    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:50:54.011604 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005952    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:50:54.011604 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005957    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:50:54.011604 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005961    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:50:54.011604 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005966    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:50:54.011604 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005971    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:50:54.011604 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005975    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:50:54.011604 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005979    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:50:54.011604 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005983    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:50:54.011604 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005987    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:50:54.011604 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005991    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:50:54.011604 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005995    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:50:54.011604 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.005999    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006003    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006007    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006012    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006016    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006020    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006024    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006028    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006032    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006036    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006040    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006044    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006048    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006052    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006057    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006061    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006065    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006070    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006074    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006078    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:50:54.011987 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006082    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006086    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006090    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006094    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006098    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006102    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006106    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006110    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006113    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006118    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006122    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006125    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006130    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006133    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006137    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006142    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006146    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006150    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006154    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006158    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:50:54.012473 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006161    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006165    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006169    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006173    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006177    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006181    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006185    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006189    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006194    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006197    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006202    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006205    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006209    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006213    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006219    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006224    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006228    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006233    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006236    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:50:54.013239 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006241    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006247    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006253    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006257    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006262    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006266    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006271    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006276    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006282    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006287    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006291    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006295    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006299    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006303    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006307    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006311    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006315    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006319    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006324    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:50:54.013855 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006328    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:50:54.014328 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006332    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:50:54.014328 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006336    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:50:54.014328 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006341    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:50:54.014328 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006345    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:50:54.014328 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006349    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:50:54.014328 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006353    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:50:54.014328 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:54.006357    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:50:54.014328 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.006364    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 02:50:54.014328 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.006498    2574 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 14 02:50:54.014328 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.009461    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 14 02:50:54.014328 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.010698    2574 server.go:1019] "Starting client certificate rotation"
Sep 14 02:50:54.014328 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.010805    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 02:50:54.014328 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.012759    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 02:50:54.016217 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.016201    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 02:50:54.017299 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.017284    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 02:50:54.028151 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.028137    2574 log.go:25] "Validated CRI v1 runtime API"
Sep 14 02:50:54.031440 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.031426    2574 log.go:25] "Validated CRI v1 image API"
Sep 14 02:50:54.032357 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.032340    2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 14 02:50:54.037245 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.037219    2574 fs.go:135] Filesystem UUIDs: map[2373b96f-4f71-4c84-9655-99ad9ee81e8f:/dev/nvme0n1p3 5dbbc58d-912d-4089-9e5a-2d3d7d4dadfd:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 14 02:50:54.037328 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.037244    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 14 02:50:54.042243 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.042133    2574 manager.go:216] Machine: {Timestamp:2026-09-14 02:50:54.040727682 +0000 UTC m=+0.353751492 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100354 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec264b485c30f53eb220b915930095f2 SystemUUID:ec264b48-5c30-f53e-b220-b915930095f2 BootID:908eef59-e0dc-4633-8dfe-7ff276069f1f Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d0:3c:f7:13 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d0:3c:f7:13 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:47:1a:51:d8:2e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 14 02:50:54.042243 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.042233    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 14 02:50:54.042395 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.042322    2574 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 14 02:50:54.045931 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.045904    2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 14 02:50:54.046055 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.045934    2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-188.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 14 02:50:54.046794 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.046785    2574 topology_manager.go:138] "Creating topology manager with none policy"
Sep 14 02:50:54.046833 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.046800    2574 container_manager_linux.go:309] "Creating device plugin manager"
Sep 14 02:50:54.046833 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.046807    2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 02:50:54.047653 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.047642    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 02:50:54.048928 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.048918    2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 14 02:50:54.054237 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.054221    2574 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 02:50:54.054368 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.054358    2574 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 14 02:50:54.054406 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.054395    2574 kubelet.go:486] "Attempting to sync node with API server"
Sep 14 02:50:54.054436 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.054408    2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 14 02:50:54.054436 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.054422    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 14 02:50:54.054436 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.054431    2574 kubelet.go:398] "Adding apiserver pod source"
Sep 14 02:50:54.054562 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.054439    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 14 02:50:54.054852 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.054839    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 02:50:54.054913 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.054864    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 02:50:54.058048 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.058023    2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 14 02:50:54.062785 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.062770    2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 14 02:50:54.062845 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.062800    2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 14 02:50:54.065608 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.065595    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 14 02:50:54.065677 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.065612    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 14 02:50:54.065677 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.065619    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 14 02:50:54.065677 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.065624    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 14 02:50:54.065677 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.065632    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 14 02:50:54.065677 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.065637    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 14 02:50:54.065677 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.065650    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 14 02:50:54.065677 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.065655    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 14 02:50:54.065677 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.065661    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 14 02:50:54.065677 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.065666    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 14 02:50:54.065677 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.065675    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 14 02:50:54.065677 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.065679    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 14 02:50:54.066599 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.066590    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 14 02:50:54.066599 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.066598    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 14 02:50:54.069889 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.069862    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 02:50:54.070408 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.070394    2574 server.go:1268] "Started kubelet"
Sep 14 02:50:54.070489 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.070460    2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 14 02:50:54.070563 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.070479    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 14 02:50:54.070604 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.070564    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 14 02:50:54.071136 ip-10-0-133-188 systemd[1]: Started Kubernetes Kubelet.
Sep 14 02:50:54.071855 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.071828    2574 server.go:310] "Adding debug handlers to kubelet server"
Sep 14 02:50:54.073662 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.073640    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-188.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 14 02:50:54.073794 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.073771    2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 14 02:50:54.076410 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.076378    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 14 02:50:54.076506 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.076448    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 14 02:50:54.078087 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.078070    2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 14 02:50:54.081080 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.081064    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 14 02:50:54.081723 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.081707    2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 14 02:50:54.082141 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.081949    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Sep 14 02:50:54.082574 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.082481    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 14 02:50:54.082574 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.082482    2574 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 14 02:50:54.082574 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.082508    2574 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 14 02:50:54.082702 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.082637    2574 reconstruct.go:97] "Volume reconstruction finished"
Sep 14 02:50:54.082702 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.082644    2574 reconciler.go:29] "Reconciler: start to sync state"
Sep 14 02:50:54.083295 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.082138    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-188.ec2.internal.18d51036b332c13e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-188.ec2.internal,UID:ip-10-0-133-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-188.ec2.internal,},FirstTimestamp:2026-09-14 02:50:54.070366526 +0000 UTC m=+0.383390335,LastTimestamp:2026-09-14 02:50:54.070366526 +0000 UTC m=+0.383390335,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-188.ec2.internal,}"
Sep 14 02:50:54.083295 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.083296    2574 factory.go:55] Registering systemd factory
Sep 14 02:50:54.083449 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.083308    2574 factory.go:223] Registration of the systemd container factory successfully
Sep 14 02:50:54.083548 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.083508    2574 factory.go:153] Registering CRI-O factory
Sep 14 02:50:54.083548 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.083550    2574 factory.go:223] Registration of the crio container factory successfully
Sep 14 02:50:54.083715 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.083607    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 14 02:50:54.083715 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.083626    2574 factory.go:103] Registering Raw factory
Sep 14 02:50:54.083715 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.083640    2574 manager.go:1196] Started watching for new ooms in manager
Sep 14 02:50:54.084208 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.084013    2574 manager.go:349] Starting recovery of all containers
Sep 14 02:50:54.087095 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.087065    2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 14 02:50:54.087217 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.087193    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 14 02:50:54.091955 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.091905    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 14 02:50:54.095461 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.095410    2574 manager.go:354] Recovery completed
Sep 14 02:50:54.101725 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.101711    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:54.103252 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.103234    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:54.103338 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.103270    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:54.103338 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.103286    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:54.103876 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.103860    2574 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 14 02:50:54.103876 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.103869    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:54.103876 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.103876    2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 14 02:50:54.103876 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.103892    2574 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 02:50:54.104574 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.104559    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:54.104643 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.104586    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:54.104643 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.104598    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:54.106020 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.106008    2574 policy_none.go:49] "None policy: Start"
Sep 14 02:50:54.106058 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.106025    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 14 02:50:54.106058 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.106035    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 14 02:50:54.107387 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.107318    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-188.ec2.internal.18d51036b5288e26  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-188.ec2.internal,UID:ip-10-0-133-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-188.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-188.ec2.internal,},FirstTimestamp:2026-09-14 02:50:54.103252518 +0000 UTC m=+0.416276334,LastTimestamp:2026-09-14 02:50:54.103252518 +0000 UTC m=+0.416276334,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-188.ec2.internal,}"
Sep 14 02:50:54.107968 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.107955    2574 policy_none.go:47] "Start"
Sep 14 02:50:54.118810 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.118745    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-188.ec2.internal.18d51036b528f024  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-188.ec2.internal,UID:ip-10-0-133-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-188.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-188.ec2.internal,},FirstTimestamp:2026-09-14 02:50:54.103277604 +0000 UTC m=+0.416301424,LastTimestamp:2026-09-14 02:50:54.103277604 +0000 UTC m=+0.416301424,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-188.ec2.internal,}"
Sep 14 02:50:54.129106 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.129046    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-188.ec2.internal.18d51036b529256b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-188.ec2.internal,UID:ip-10-0-133-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-188.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-188.ec2.internal,},FirstTimestamp:2026-09-14 02:50:54.103291243 +0000 UTC m=+0.416315054,LastTimestamp:2026-09-14 02:50:54.103291243 +0000 UTC m=+0.416315054,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-188.ec2.internal,}"
Sep 14 02:50:54.163583 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.145223    2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-188.ec2.internal.18d51036b5288e26\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-188.ec2.internal.18d51036b5288e26  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-188.ec2.internal,UID:ip-10-0-133-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-188.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-188.ec2.internal,},FirstTimestamp:2026-09-14 02:50:54.103252518 +0000 UTC m=+0.416276334,LastTimestamp:2026-09-14 02:50:54.104573028 +0000 UTC m=+0.417596839,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-188.ec2.internal,}"
Sep 14 02:50:54.163583 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.153029    2574 manager.go:337] "Starting Device Plugin manager"
Sep 14 02:50:54.163583 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.153062    2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 14 02:50:54.163583 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.153070    2574 server.go:87] "Starting device plugin registration server"
Sep 14 02:50:54.163583 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.153264    2574 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 14 02:50:54.163583 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.153274    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 14 02:50:54.163583 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.153381    2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 14 02:50:54.163583 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.153470    2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 14 02:50:54.163583 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.153479    2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 14 02:50:54.163583 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.153945    2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 14 02:50:54.163583 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.153986    2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-188.ec2.internal\" not found"
Sep 14 02:50:54.163583 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.157208    2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-188.ec2.internal.18d51036b528f024\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-188.ec2.internal.18d51036b528f024  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-188.ec2.internal,UID:ip-10-0-133-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-188.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-188.ec2.internal,},FirstTimestamp:2026-09-14 02:50:54.103277604 +0000 UTC m=+0.416301424,LastTimestamp:2026-09-14 02:50:54.10459326 +0000 UTC m=+0.417617069,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-188.ec2.internal,}"
Sep 14 02:50:54.171271 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.171215    2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-188.ec2.internal.18d51036b529256b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-188.ec2.internal.18d51036b529256b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-188.ec2.internal,UID:ip-10-0-133-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-188.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-188.ec2.internal,},FirstTimestamp:2026-09-14 02:50:54.103291243 +0000 UTC m=+0.416315054,LastTimestamp:2026-09-14 02:50:54.104602126 +0000 UTC m=+0.417625937,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-188.ec2.internal,}"
Sep 14 02:50:54.187886 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.187824    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-188.ec2.internal.18d51036b84367bf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-188.ec2.internal,UID:ip-10-0-133-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-133-188.ec2.internal,},FirstTimestamp:2026-09-14 02:50:54.155343807 +0000 UTC m=+0.468367604,LastTimestamp:2026-09-14 02:50:54.155343807 +0000 UTC m=+0.468367604,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-188.ec2.internal,}"
Sep 14 02:50:54.209145 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.209128    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 14 02:50:54.209223 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.209171    2574 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 14 02:50:54.209223 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.209191    2574 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 14 02:50:54.209327 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.209221    2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 14 02:50:54.231120 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.231092    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 14 02:50:54.251009 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.250971    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8svrm"
Sep 14 02:50:54.254075 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.254053    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:54.254806 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.254791    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:54.254883 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.254818    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:54.254883 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.254829    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:54.254883 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.254848    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.268050 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.267990    2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-188.ec2.internal.18d51036b5288e26\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-188.ec2.internal.18d51036b5288e26  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-188.ec2.internal,UID:ip-10-0-133-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-188.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-188.ec2.internal,},FirstTimestamp:2026-09-14 02:50:54.103252518 +0000 UTC m=+0.416276334,LastTimestamp:2026-09-14 02:50:54.254806229 +0000 UTC m=+0.567830038,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-188.ec2.internal,}"
Sep 14 02:50:54.271965 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.271951    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8svrm"
Sep 14 02:50:54.272574 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.272543    2574 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-133-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.272654 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.272595    2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-188.ec2.internal.18d51036b528f024\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-188.ec2.internal.18d51036b528f024  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-188.ec2.internal,UID:ip-10-0-133-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-188.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-188.ec2.internal,},FirstTimestamp:2026-09-14 02:50:54.103277604 +0000 UTC m=+0.416301424,LastTimestamp:2026-09-14 02:50:54.254823202 +0000 UTC m=+0.567847012,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-188.ec2.internal,}"
Sep 14 02:50:54.281610 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.281545    2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-188.ec2.internal.18d51036b529256b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-188.ec2.internal.18d51036b529256b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-188.ec2.internal,UID:ip-10-0-133-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-188.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-188.ec2.internal,},FirstTimestamp:2026-09-14 02:50:54.103291243 +0000 UTC m=+0.416315054,LastTimestamp:2026-09-14 02:50:54.254833051 +0000 UTC m=+0.567856862,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-188.ec2.internal,}"
Sep 14 02:50:54.295610 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.295592    2574 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-188.ec2.internal\" not found" node="ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.310202 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.310182    2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal"]
Sep 14 02:50:54.310249 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.310238    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:54.310985 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.310971    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:54.311036 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.310995    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:54.311036 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.311008    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:54.313252 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.313240    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:54.313365 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.313350    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.313415 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.313378    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:54.313932 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.313918    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:54.313988 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.313947    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:54.313988 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.313956    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:54.314046 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.313917    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:54.314046 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.314027    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:54.314046 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.314039    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:54.316239 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.316224    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.316286 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.316250    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:54.316913 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.316894    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:54.317004 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.316931    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:54.317004 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.316948    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:54.349133 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.349115    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-188.ec2.internal\" not found" node="ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.353332 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.353317    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-188.ec2.internal\" not found" node="ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.383856 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.383835    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ab003f2b739da2d03a150c7e3dc691ab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal\" (UID: \"ab003f2b739da2d03a150c7e3dc691ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.383932 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.383865    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab003f2b739da2d03a150c7e3dc691ab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal\" (UID: \"ab003f2b739da2d03a150c7e3dc691ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.383932 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.383889    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/07e0d660232ee4077dd70b8b3442d37b-config\") pod \"kube-apiserver-proxy-ip-10-0-133-188.ec2.internal\" (UID: \"07e0d660232ee4077dd70b8b3442d37b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.472891 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.472864    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:54.473689 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.473672    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:54.473762 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.473704    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:54.473762 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.473716    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:54.473762 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.473736    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.483971 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.483953    2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.484013 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.483971    2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-188.ec2.internal\": node \"ip-10-0-133-188.ec2.internal\" not found"
Sep 14 02:50:54.484077 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.484062    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ab003f2b739da2d03a150c7e3dc691ab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal\" (UID: \"ab003f2b739da2d03a150c7e3dc691ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.484113 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.484086    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab003f2b739da2d03a150c7e3dc691ab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal\" (UID: \"ab003f2b739da2d03a150c7e3dc691ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.484113 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.484103    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/07e0d660232ee4077dd70b8b3442d37b-config\") pod \"kube-apiserver-proxy-ip-10-0-133-188.ec2.internal\" (UID: \"07e0d660232ee4077dd70b8b3442d37b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.484202 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.484135    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/07e0d660232ee4077dd70b8b3442d37b-config\") pod \"kube-apiserver-proxy-ip-10-0-133-188.ec2.internal\" (UID: \"07e0d660232ee4077dd70b8b3442d37b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.484202 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.484144    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ab003f2b739da2d03a150c7e3dc691ab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal\" (UID: \"ab003f2b739da2d03a150c7e3dc691ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.484202 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.484157    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab003f2b739da2d03a150c7e3dc691ab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal\" (UID: \"ab003f2b739da2d03a150c7e3dc691ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.519409 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.519363    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Sep 14 02:50:54.619910 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.619880    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Sep 14 02:50:54.652053 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.652030    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.655797 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:54.655781    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal"
Sep 14 02:50:54.720982 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.720839    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Sep 14 02:50:54.821351 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.821296    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Sep 14 02:50:54.921834 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:54.921813    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Sep 14 02:50:55.010455 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.010427    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 14 02:50:55.022706 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.022689    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-188.ec2.internal\" not found"
Sep 14 02:50:55.036551 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.036534    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 02:50:55.055049 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.055031    2574 apiserver.go:52] "Watching apiserver"
Sep 14 02:50:55.065794 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.065774    2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 14 02:50:55.066674 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.066656    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-kklhq","kube-system/global-pull-secret-syncer-wdp4n","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62","openshift-dns/node-resolver-vfx28","openshift-multus/multus-5mpdv","openshift-network-diagnostics/network-check-target-5jzxs","openshift-ovn-kubernetes/ovnkube-node-d4jxp","kube-system/konnectivity-agent-nsgwq","openshift-cluster-node-tuning-operator/tuned-4lbmj","openshift-image-registry/node-ca-84ls2","openshift-multus/multus-additional-cni-plugins-54ht7","openshift-multus/network-metrics-daemon-hkpjq"]
Sep 14 02:50:55.069132 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.069117    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.072222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.072174    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.073426 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.073411    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 14 02:50:55.073863 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.073850    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 14 02:50:55.073948 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.073933    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 14 02:50:55.074148 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.074134    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7lpxs\""
Sep 14 02:50:55.074588 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.074575    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vxp5l\""
Sep 14 02:50:55.074780 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.074763    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 14 02:50:55.074819 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.074767    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 14 02:50:55.075161 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.075146    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:50:55.075213 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.075197    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:50:55.075458 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.075447    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 14 02:50:55.075922 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.075907    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 14 02:50:55.076253 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.076239    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 14 02:50:55.076567 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.076554    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 14 02:50:55.077945 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.077930    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kklhq"
Sep 14 02:50:55.080585 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.080570    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:50:55.080681 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.080635    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:50:55.080774 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.080758    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 14 02:50:55.080827 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.080817    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 14 02:50:55.081162 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.081148    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 14 02:50:55.081221 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.081173    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 14 02:50:55.081379 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.081365    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tphq9\""
Sep 14 02:50:55.082428 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.082413    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal"
Sep 14 02:50:55.083201 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.083182    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vfx28"
Sep 14 02:50:55.085930 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.085912    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.087386 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087369    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwdkd\" (UniqueName: \"kubernetes.io/projected/60ad1800-10ba-4d33-a609-cb0e66dfdac5-kube-api-access-wwdkd\") pod \"node-resolver-vfx28\" (UID: \"60ad1800-10ba-4d33-a609-cb0e66dfdac5\") " pod="openshift-dns/node-resolver-vfx28"
Sep 14 02:50:55.087617 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087392    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ffab345-9789-435a-b2ae-b53ec8e66f43-host-slash\") pod \"iptables-alerter-kklhq\" (UID: \"1ffab345-9789-435a-b2ae-b53ec8e66f43\") " pod="openshift-network-operator/iptables-alerter-kklhq"
Sep 14 02:50:55.087617 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087406    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2a5808dc-86c4-4507-8281-09e0c2910f77-dbus\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:50:55.087617 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087420    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.087617 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087467    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9fr4\" (UniqueName: \"kubernetes.io/projected/bc4edffb-356b-461d-ad74-52b9e26e243d-kube-api-access-v9fr4\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.087617 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087486    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-log-socket\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.087617 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087532    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6608dc19-84cb-4438-8940-38f32c25381c-ovn-node-metrics-cert\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.087617 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087574    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqm95\" (UniqueName: \"kubernetes.io/projected/1ffab345-9789-435a-b2ae-b53ec8e66f43-kube-api-access-cqm95\") pod \"iptables-alerter-kklhq\" (UID: \"1ffab345-9789-435a-b2ae-b53ec8e66f43\") " pod="openshift-network-operator/iptables-alerter-kklhq"
Sep 14 02:50:55.087617 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087601    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-kubelet\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.087941 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087633    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-var-lib-openvswitch\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.087941 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087661    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-run-openvswitch\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.087941 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087682    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-slash\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.087941 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087702    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-node-log\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.087941 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087731    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-run-ovn-kubernetes\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.087941 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087749    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6608dc19-84cb-4438-8940-38f32c25381c-env-overrides\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.087941 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087781    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvqb7\" (UniqueName: \"kubernetes.io/projected/6608dc19-84cb-4438-8940-38f32c25381c-kube-api-access-gvqb7\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.087941 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087808    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-registration-dir\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.087941 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087833    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-run-systemd\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.087941 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087849    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-etc-openvswitch\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.087941 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087862    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-run-ovn\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.087941 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087903    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60ad1800-10ba-4d33-a609-cb0e66dfdac5-tmp-dir\") pod \"node-resolver-vfx28\" (UID: \"60ad1800-10ba-4d33-a609-cb0e66dfdac5\") " pod="openshift-dns/node-resolver-vfx28"
Sep 14 02:50:55.087941 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087935    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1ffab345-9789-435a-b2ae-b53ec8e66f43-iptables-alerter-script\") pod \"iptables-alerter-kklhq\" (UID: \"1ffab345-9789-435a-b2ae-b53ec8e66f43\") " pod="openshift-network-operator/iptables-alerter-kklhq"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.087990    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088051    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/60ad1800-10ba-4d33-a609-cb0e66dfdac5-hosts-file\") pod \"node-resolver-vfx28\" (UID: \"60ad1800-10ba-4d33-a609-cb0e66dfdac5\") " pod="openshift-dns/node-resolver-vfx28"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088076    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-sys-fs\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088101    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-run-netns\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088127    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6608dc19-84cb-4438-8940-38f32c25381c-ovnkube-config\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088156    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l7tv\" (UniqueName: \"kubernetes.io/projected/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-kube-api-access-4l7tv\") pod \"network-metrics-daemon-hkpjq\" (UID: \"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6\") " pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088181    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088215    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs\") pod \"network-metrics-daemon-hkpjq\" (UID: \"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6\") " pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088240    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-systemd-units\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088260    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-cni-netd\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088273    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2a5808dc-86c4-4507-8281-09e0c2910f77-kubelet-config\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088293    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-socket-dir\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088315    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-device-dir\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088335    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-etc-selinux\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088349    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-cni-bin\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088400    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6608dc19-84cb-4438-8940-38f32c25381c-ovnkube-script-lib\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.088619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088529    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 14 02:50:55.089255 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088502    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 14 02:50:55.089255 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088553    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 14 02:50:55.089255 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088647    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 14 02:50:55.089255 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088648    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pzw52\""
Sep 14 02:50:55.089255 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088718    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 14 02:50:55.089255 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088775    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 14 02:50:55.089255 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.088801    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:50:55.089255 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.088864    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:50:55.089589 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.089556    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ck67h\""
Sep 14 02:50:55.091602 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.091585    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nsgwq"
Sep 14 02:50:55.094452 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.094435    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.096659 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.096646    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 14 02:50:55.096836 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.096821    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-84ls2"
Sep 14 02:50:55.096925 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.096911    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7tlp9\""
Sep 14 02:50:55.097396 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.097379    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 14 02:50:55.097465 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.097452    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-72hdt\""
Sep 14 02:50:55.098707 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.098692    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 14 02:50:55.099097 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.099082    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.100613 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.100597    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 14 02:50:55.100981 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.100968    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 14 02:50:55.101574 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.101562    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 14 02:50:55.101870 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.101855    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s99n5\""
Sep 14 02:50:55.102027 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.102012    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 14 02:50:55.102216 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.102202    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fd9hg\""
Sep 14 02:50:55.102283 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.102225    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 14 02:50:55.104699 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.104680    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 14 02:50:55.108385 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.108369    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 02:50:55.108477 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.108427    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"
Sep 14 02:50:55.108552 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.108538    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal"]
Sep 14 02:50:55.113896 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.113881    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 02:50:55.131345 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.131329    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 02:50:55.133981 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.133580    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal"]
Sep 14 02:50:55.150726 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.150708    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8794l"
Sep 14 02:50:55.168270 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.168251    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8794l"
Sep 14 02:50:55.184232 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.184214    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 14 02:50:55.189159 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189144    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-slash\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.189221 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189169    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a851c347-abe6-4dce-8b64-5b40a3a812a8-serviceca\") pod \"node-ca-84ls2\" (UID: \"a851c347-abe6-4dce-8b64-5b40a3a812a8\") " pod="openshift-image-registry/node-ca-84ls2"
Sep 14 02:50:55.189221 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189185    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-cni-binary-copy\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.189221 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189200    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-multus-conf-dir\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.189221 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189213    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-lib-modules\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.189346 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189267    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-slash\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.189346 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189304    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-tmp\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.189346 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189329    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-registration-dir\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.189462 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189351    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-run-systemd\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.189462 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189372    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-etc-openvswitch\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.189462 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189394    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-run-ovn\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.189462 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189404    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-run-systemd\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.189462 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189412    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-registration-dir\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.189462 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189418    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b970a991-aaae-40da-9abb-4a84d3d53b61-cnibin\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.189462 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189445    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-run-ovn\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.189462 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189423    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-etc-openvswitch\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.189462 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189453    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60ad1800-10ba-4d33-a609-cb0e66dfdac5-tmp-dir\") pod \"node-resolver-vfx28\" (UID: \"60ad1800-10ba-4d33-a609-cb0e66dfdac5\") " pod="openshift-dns/node-resolver-vfx28"
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189478    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189495    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-sys-fs\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189541    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-run-netns\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189558    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-sys-fs\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189565    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6608dc19-84cb-4438-8940-38f32c25381c-ovnkube-config\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189591    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a9bb9b8-0a23-445a-9074-930df7d80ea0-konnectivity-ca\") pod \"konnectivity-agent-nsgwq\" (UID: \"1a9bb9b8-0a23-445a-9074-930df7d80ea0\") " pod="kube-system/konnectivity-agent-nsgwq"
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189598    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-run-netns\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.189617    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.189696    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret podName:2a5808dc-86c4-4507-8281-09e0c2910f77 nodeName:}" failed. No retries permitted until 2026-09-14 02:50:55.689663669 +0000 UTC m=+2.002687480 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret") pod "global-pull-secret-syncer-wdp4n" (UID: "2a5808dc-86c4-4507-8281-09e0c2910f77") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189718    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b970a991-aaae-40da-9abb-4a84d3d53b61-cni-binary-copy\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189724    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60ad1800-10ba-4d33-a609-cb0e66dfdac5-tmp-dir\") pod \"node-resolver-vfx28\" (UID: \"60ad1800-10ba-4d33-a609-cb0e66dfdac5\") " pod="openshift-dns/node-resolver-vfx28"
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189747    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l7tv\" (UniqueName: \"kubernetes.io/projected/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-kube-api-access-4l7tv\") pod \"network-metrics-daemon-hkpjq\" (UID: \"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6\") " pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189774    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-cnibin\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189794    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-var-lib-cni-multus\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189809    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-run\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189838    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-var-lib-kubelet\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.189865 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189857    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs\") pod \"network-metrics-daemon-hkpjq\" (UID: \"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6\") " pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189871    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-systemd-units\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189902    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-cni-netd\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189926    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-systemd-units\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.189941    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189932    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm4bt\" (UniqueName: \"kubernetes.io/projected/a851c347-abe6-4dce-8b64-5b40a3a812a8-kube-api-access-nm4bt\") pod \"node-ca-84ls2\" (UID: \"a851c347-abe6-4dce-8b64-5b40a3a812a8\") " pod="openshift-image-registry/node-ca-84ls2"
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189964    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-cni-netd\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.189987    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvv24\" (UniqueName: \"kubernetes.io/projected/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-kube-api-access-zvv24\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.190004    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs podName:ae964ae5-7b42-42c5-8b19-1bf113cfd6d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:50:55.689991341 +0000 UTC m=+2.003015138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs") pod "network-metrics-daemon-hkpjq" (UID: "ae964ae5-7b42-42c5-8b19-1bf113cfd6d6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190027    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-systemd\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190046    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2a5808dc-86c4-4507-8281-09e0c2910f77-kubelet-config\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190061    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-socket-dir\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190072    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2a5808dc-86c4-4507-8281-09e0c2910f77-kubelet-config\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190075    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-device-dir\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190104    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-device-dir\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190103    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6608dc19-84cb-4438-8940-38f32c25381c-ovnkube-script-lib\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190130    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6608dc19-84cb-4438-8940-38f32c25381c-ovnkube-config\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.190628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190155    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-socket-dir\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190133    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-run-k8s-cni-cncf-io\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-run-multus-certs\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190245    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-kubernetes\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190267    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-sysctl-conf\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190294    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwdkd\" (UniqueName: \"kubernetes.io/projected/60ad1800-10ba-4d33-a609-cb0e66dfdac5-kube-api-access-wwdkd\") pod \"node-resolver-vfx28\" (UID: \"60ad1800-10ba-4d33-a609-cb0e66dfdac5\") " pod="openshift-dns/node-resolver-vfx28"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190319    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ffab345-9789-435a-b2ae-b53ec8e66f43-host-slash\") pod \"iptables-alerter-kklhq\" (UID: \"1ffab345-9789-435a-b2ae-b53ec8e66f43\") " pod="openshift-network-operator/iptables-alerter-kklhq"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190344    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190366    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-log-socket\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190404    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190402    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ffab345-9789-435a-b2ae-b53ec8e66f43-host-slash\") pod \"iptables-alerter-kklhq\" (UID: \"1ffab345-9789-435a-b2ae-b53ec8e66f43\") " pod="openshift-network-operator/iptables-alerter-kklhq"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190417    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-log-socket\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190434    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6608dc19-84cb-4438-8940-38f32c25381c-ovn-node-metrics-cert\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190468    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb95b\" (UniqueName: \"kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b\") pod \"network-check-target-5jzxs\" (UID: \"1d1732d2-7663-4622-bc8e-92f5c6d6555a\") " pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190491    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6608dc19-84cb-4438-8940-38f32c25381c-ovnkube-script-lib\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190507    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-hostroot\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190541    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-sys\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.191409 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190558    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqm95\" (UniqueName: \"kubernetes.io/projected/1ffab345-9789-435a-b2ae-b53ec8e66f43-kube-api-access-cqm95\") pod \"iptables-alerter-kklhq\" (UID: \"1ffab345-9789-435a-b2ae-b53ec8e66f43\") " pod="openshift-network-operator/iptables-alerter-kklhq"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190612    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-var-lib-openvswitch\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190645    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-run-openvswitch\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190656    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-var-lib-openvswitch\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190663    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-etc-kubernetes\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190701    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-sysctl-d\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190711    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-run-openvswitch\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190723    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-tuned\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190758    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-node-log\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190783    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-run-ovn-kubernetes\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190803    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-node-log\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190805    2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190810    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6608dc19-84cb-4438-8940-38f32c25381c-env-overrides\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190848    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvqb7\" (UniqueName: \"kubernetes.io/projected/6608dc19-84cb-4438-8940-38f32c25381c-kube-api-access-gvqb7\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190863    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-run-ovn-kubernetes\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190874    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmbsm\" (UniqueName: \"kubernetes.io/projected/b970a991-aaae-40da-9abb-4a84d3d53b61-kube-api-access-vmbsm\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.190984    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-multus-cni-dir\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.192105 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191009    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-multus-daemon-config\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191031    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-sysconfig\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191055    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-var-lib-cni-bin\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191082    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1ffab345-9789-435a-b2ae-b53ec8e66f43-iptables-alerter-script\") pod \"iptables-alerter-kklhq\" (UID: \"1ffab345-9789-435a-b2ae-b53ec8e66f43\") " pod="openshift-network-operator/iptables-alerter-kklhq"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191106    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/60ad1800-10ba-4d33-a609-cb0e66dfdac5-hosts-file\") pod \"node-resolver-vfx28\" (UID: \"60ad1800-10ba-4d33-a609-cb0e66dfdac5\") " pod="openshift-dns/node-resolver-vfx28"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191122    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a851c347-abe6-4dce-8b64-5b40a3a812a8-host\") pod \"node-ca-84ls2\" (UID: \"a851c347-abe6-4dce-8b64-5b40a3a812a8\") " pod="openshift-image-registry/node-ca-84ls2"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191137    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b970a991-aaae-40da-9abb-4a84d3d53b61-system-cni-dir\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191151    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-var-lib-kubelet\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191167    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191189    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a9bb9b8-0a23-445a-9074-930df7d80ea0-agent-certs\") pod \"konnectivity-agent-nsgwq\" (UID: \"1a9bb9b8-0a23-445a-9074-930df7d80ea0\") " pod="kube-system/konnectivity-agent-nsgwq"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191199    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6608dc19-84cb-4438-8940-38f32c25381c-env-overrides\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191231    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b970a991-aaae-40da-9abb-4a84d3d53b61-tuning-conf-dir\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191242    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/60ad1800-10ba-4d33-a609-cb0e66dfdac5-hosts-file\") pod \"node-resolver-vfx28\" (UID: \"60ad1800-10ba-4d33-a609-cb0e66dfdac5\") " pod="openshift-dns/node-resolver-vfx28"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191243    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191257    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx9lb\" (UniqueName: \"kubernetes.io/projected/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-kube-api-access-kx9lb\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191286    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-multus-socket-dir-parent\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191307    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-etc-selinux\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.192769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191321    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-cni-bin\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.193453 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191341    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-run-netns\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.193453 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191363    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-modprobe-d\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.193453 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191388    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2a5808dc-86c4-4507-8281-09e0c2910f77-dbus\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:50:55.193453 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191390    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-cni-bin\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.193453 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191411    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9fr4\" (UniqueName: \"kubernetes.io/projected/bc4edffb-356b-461d-ad74-52b9e26e243d-kube-api-access-v9fr4\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.193453 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191452    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b970a991-aaae-40da-9abb-4a84d3d53b61-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.193453 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191477    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b970a991-aaae-40da-9abb-4a84d3d53b61-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.193453 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191485    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc4edffb-356b-461d-ad74-52b9e26e243d-etc-selinux\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.193453 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191502    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-system-cni-dir\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.193453 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191541    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-host\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.193453 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191572    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-kubelet\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.193453 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191616    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1ffab345-9789-435a-b2ae-b53ec8e66f43-iptables-alerter-script\") pod \"iptables-alerter-kklhq\" (UID: \"1ffab345-9789-435a-b2ae-b53ec8e66f43\") " pod="openshift-network-operator/iptables-alerter-kklhq"
Sep 14 02:50:55.193453 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191649    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6608dc19-84cb-4438-8940-38f32c25381c-host-kubelet\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.193453 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.191837    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2a5808dc-86c4-4507-8281-09e0c2910f77-dbus\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:50:55.194052 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.193932    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 02:50:55.194562 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.194541    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6608dc19-84cb-4438-8940-38f32c25381c-ovn-node-metrics-cert\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.209386 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:55.209361    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07e0d660232ee4077dd70b8b3442d37b.slice/crio-0764b37a3bd59a69942d76db1a42b1ae30c27337fe4c84e6f0f67c2a057e572a WatchSource:0}: Error finding container 0764b37a3bd59a69942d76db1a42b1ae30c27337fe4c84e6f0f67c2a057e572a: Status 404 returned error can't find the container with id 0764b37a3bd59a69942d76db1a42b1ae30c27337fe4c84e6f0f67c2a057e572a
Sep 14 02:50:55.212474 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.212449    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwdkd\" (UniqueName: \"kubernetes.io/projected/60ad1800-10ba-4d33-a609-cb0e66dfdac5-kube-api-access-wwdkd\") pod \"node-resolver-vfx28\" (UID: \"60ad1800-10ba-4d33-a609-cb0e66dfdac5\") " pod="openshift-dns/node-resolver-vfx28"
Sep 14 02:50:55.212585 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.212470    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvqb7\" (UniqueName: \"kubernetes.io/projected/6608dc19-84cb-4438-8940-38f32c25381c-kube-api-access-gvqb7\") pod \"ovnkube-node-d4jxp\" (UID: \"6608dc19-84cb-4438-8940-38f32c25381c\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.214891 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.214870    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9fr4\" (UniqueName: \"kubernetes.io/projected/bc4edffb-356b-461d-ad74-52b9e26e243d-kube-api-access-v9fr4\") pod \"aws-ebs-csi-driver-node-rhj62\" (UID: \"bc4edffb-356b-461d-ad74-52b9e26e243d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.216089 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.216067    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l7tv\" (UniqueName: \"kubernetes.io/projected/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-kube-api-access-4l7tv\") pod \"network-metrics-daemon-hkpjq\" (UID: \"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6\") " pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:50:55.216306 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.216286    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqm95\" (UniqueName: \"kubernetes.io/projected/1ffab345-9789-435a-b2ae-b53ec8e66f43-kube-api-access-cqm95\") pod \"iptables-alerter-kklhq\" (UID: \"1ffab345-9789-435a-b2ae-b53ec8e66f43\") " pod="openshift-network-operator/iptables-alerter-kklhq"
Sep 14 02:50:55.223928 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.223913    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 02:50:55.231079 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:55.231051    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab003f2b739da2d03a150c7e3dc691ab.slice/crio-6fecaab152855ecff4cbc0ac1ad3a7ccbf38f41d525a8572d3f81919e1454a4b WatchSource:0}: Error finding container 6fecaab152855ecff4cbc0ac1ad3a7ccbf38f41d525a8572d3f81919e1454a4b: Status 404 returned error can't find the container with id 6fecaab152855ecff4cbc0ac1ad3a7ccbf38f41d525a8572d3f81919e1454a4b
Sep 14 02:50:55.276217 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.276161    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-13 02:45:54 +0000 UTC" deadline="2028-06-03 11:37:58.729364778 +0000 UTC"
Sep 14 02:50:55.276217 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.276213    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15080h47m3.45315535s"
Sep 14 02:50:55.292437 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292413    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-cnibin\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.292555 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292443    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-var-lib-cni-multus\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.292555 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292466    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-run\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.292555 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292489    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-var-lib-kubelet\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.292555 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292542    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-cnibin\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.292751 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292555    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-var-lib-cni-multus\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.292751 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292564    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-run\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.292751 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292595    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm4bt\" (UniqueName: \"kubernetes.io/projected/a851c347-abe6-4dce-8b64-5b40a3a812a8-kube-api-access-nm4bt\") pod \"node-ca-84ls2\" (UID: \"a851c347-abe6-4dce-8b64-5b40a3a812a8\") " pod="openshift-image-registry/node-ca-84ls2"
Sep 14 02:50:55.292751 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292586    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-var-lib-kubelet\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.292751 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292622    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvv24\" (UniqueName: \"kubernetes.io/projected/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-kube-api-access-zvv24\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.292751 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292638    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-systemd\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.292751 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292655    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-run-k8s-cni-cncf-io\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.292751 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292670    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-run-multus-certs\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.292751 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292688    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-kubernetes\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.292751 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292706    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-sysctl-conf\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.292751 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292717    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-systemd\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.292751 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292726    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-run-k8s-cni-cncf-io\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292768    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-run-multus-certs\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292787    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-kubernetes\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292845    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-sysctl-conf\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292871    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb95b\" (UniqueName: \"kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b\") pod \"network-check-target-5jzxs\" (UID: \"1d1732d2-7663-4622-bc8e-92f5c6d6555a\") " pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292898    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-hostroot\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292912    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-sys\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292927    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-etc-kubernetes\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292949    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-sysctl-d\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292964    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-tuned\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.292981    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmbsm\" (UniqueName: \"kubernetes.io/projected/b970a991-aaae-40da-9abb-4a84d3d53b61-kube-api-access-vmbsm\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293003    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-sys\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293006    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-multus-cni-dir\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293041    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-multus-daemon-config\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293057    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-sysconfig\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293063    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-multus-cni-dir\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293092    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-sysctl-d\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293006    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-hostroot\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.293356 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293015    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-etc-kubernetes\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293106    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-var-lib-cni-bin\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293136    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-var-lib-cni-bin\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293143    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-sysconfig\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293148    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a851c347-abe6-4dce-8b64-5b40a3a812a8-host\") pod \"node-ca-84ls2\" (UID: \"a851c347-abe6-4dce-8b64-5b40a3a812a8\") " pod="openshift-image-registry/node-ca-84ls2"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293181    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b970a991-aaae-40da-9abb-4a84d3d53b61-system-cni-dir\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293187    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a851c347-abe6-4dce-8b64-5b40a3a812a8-host\") pod \"node-ca-84ls2\" (UID: \"a851c347-abe6-4dce-8b64-5b40a3a812a8\") " pod="openshift-image-registry/node-ca-84ls2"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293210    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-var-lib-kubelet\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293230    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b970a991-aaae-40da-9abb-4a84d3d53b61-system-cni-dir\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293238    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a9bb9b8-0a23-445a-9074-930df7d80ea0-agent-certs\") pod \"konnectivity-agent-nsgwq\" (UID: \"1a9bb9b8-0a23-445a-9074-930df7d80ea0\") " pod="kube-system/konnectivity-agent-nsgwq"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293263    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b970a991-aaae-40da-9abb-4a84d3d53b61-tuning-conf-dir\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293260    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-var-lib-kubelet\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293288    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx9lb\" (UniqueName: \"kubernetes.io/projected/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-kube-api-access-kx9lb\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293314    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-multus-socket-dir-parent\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293348    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-run-netns\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293372    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-modprobe-d\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293400    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b970a991-aaae-40da-9abb-4a84d3d53b61-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.294387 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293422    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-multus-socket-dir-parent\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293436    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b970a991-aaae-40da-9abb-4a84d3d53b61-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293428    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b970a991-aaae-40da-9abb-4a84d3d53b61-tuning-conf-dir\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293507    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-host-run-netns\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293532    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-modprobe-d\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293566    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-system-cni-dir\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293620    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-host\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293645    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a851c347-abe6-4dce-8b64-5b40a3a812a8-serviceca\") pod \"node-ca-84ls2\" (UID: \"a851c347-abe6-4dce-8b64-5b40a3a812a8\") " pod="openshift-image-registry/node-ca-84ls2"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293669    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-cni-binary-copy\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293682    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-system-cni-dir\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293691    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-multus-conf-dir\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293716    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-multus-daemon-config\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293738    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-host\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293740    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-multus-conf-dir\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293775    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-lib-modules\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293800    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-tmp\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293828    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b970a991-aaae-40da-9abb-4a84d3d53b61-cnibin\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293866    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a9bb9b8-0a23-445a-9074-930df7d80ea0-konnectivity-ca\") pod \"konnectivity-agent-nsgwq\" (UID: \"1a9bb9b8-0a23-445a-9074-930df7d80ea0\") " pod="kube-system/konnectivity-agent-nsgwq"
Sep 14 02:50:55.295222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.293897    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b970a991-aaae-40da-9abb-4a84d3d53b61-cni-binary-copy\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.295998 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.294018    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b970a991-aaae-40da-9abb-4a84d3d53b61-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.295998 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.294183    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b970a991-aaae-40da-9abb-4a84d3d53b61-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.295998 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.294204    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a851c347-abe6-4dce-8b64-5b40a3a812a8-serviceca\") pod \"node-ca-84ls2\" (UID: \"a851c347-abe6-4dce-8b64-5b40a3a812a8\") " pod="openshift-image-registry/node-ca-84ls2"
Sep 14 02:50:55.295998 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.294261    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b970a991-aaae-40da-9abb-4a84d3d53b61-cnibin\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.295998 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.294303    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-lib-modules\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.295998 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.294381    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b970a991-aaae-40da-9abb-4a84d3d53b61-cni-binary-copy\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.295998 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.294395    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-cni-binary-copy\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.295998 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.294666    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1a9bb9b8-0a23-445a-9074-930df7d80ea0-konnectivity-ca\") pod \"konnectivity-agent-nsgwq\" (UID: \"1a9bb9b8-0a23-445a-9074-930df7d80ea0\") " pod="kube-system/konnectivity-agent-nsgwq"
Sep 14 02:50:55.295998 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.295588    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-etc-tuned\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.295998 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.295857    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1a9bb9b8-0a23-445a-9074-930df7d80ea0-agent-certs\") pod \"konnectivity-agent-nsgwq\" (UID: \"1a9bb9b8-0a23-445a-9074-930df7d80ea0\") " pod="kube-system/konnectivity-agent-nsgwq"
Sep 14 02:50:55.296267 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.296027    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-tmp\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.311805 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.311782    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:50:55.311889 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.311807    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:50:55.311889 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.311820    2574 projected.go:196] Error preparing data for projected volume kube-api-access-qb95b for pod openshift-network-diagnostics/network-check-target-5jzxs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:55.311889 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.311885    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b podName:1d1732d2-7663-4622-bc8e-92f5c6d6555a nodeName:}" failed. No retries permitted until 2026-09-14 02:50:55.811869356 +0000 UTC m=+2.124893170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qb95b" (UniqueName: "kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b") pod "network-check-target-5jzxs" (UID: "1d1732d2-7663-4622-bc8e-92f5c6d6555a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:55.313838 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.313797    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm4bt\" (UniqueName: \"kubernetes.io/projected/a851c347-abe6-4dce-8b64-5b40a3a812a8-kube-api-access-nm4bt\") pod \"node-ca-84ls2\" (UID: \"a851c347-abe6-4dce-8b64-5b40a3a812a8\") " pod="openshift-image-registry/node-ca-84ls2"
Sep 14 02:50:55.314001 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.313976    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmbsm\" (UniqueName: \"kubernetes.io/projected/b970a991-aaae-40da-9abb-4a84d3d53b61-kube-api-access-vmbsm\") pod \"multus-additional-cni-plugins-54ht7\" (UID: \"b970a991-aaae-40da-9abb-4a84d3d53b61\") " pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.315607 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.315588    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvv24\" (UniqueName: \"kubernetes.io/projected/4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45-kube-api-access-zvv24\") pod \"multus-5mpdv\" (UID: \"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45\") " pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.315687 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.315618    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx9lb\" (UniqueName: \"kubernetes.io/projected/d96e1f18-c6a1-47ff-96c5-a5954b361ff9-kube-api-access-kx9lb\") pod \"tuned-4lbmj\" (UID: \"d96e1f18-c6a1-47ff-96c5-a5954b361ff9\") " pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.395273 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.395211    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62"
Sep 14 02:50:55.404693 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:55.404673    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc4edffb_356b_461d_ad74_52b9e26e243d.slice/crio-c1145752e71572c9e7699bc63767b9275966b29ab90c1b54881b0b92c4b7757e WatchSource:0}: Error finding container c1145752e71572c9e7699bc63767b9275966b29ab90c1b54881b0b92c4b7757e: Status 404 returned error can't find the container with id c1145752e71572c9e7699bc63767b9275966b29ab90c1b54881b0b92c4b7757e
Sep 14 02:50:55.414292 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.414269    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:50:55.422326 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:55.422297    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6608dc19_84cb_4438_8940_38f32c25381c.slice/crio-a5cdcfbf87e92b788d6f97f2cb192594f03da06048eb1fbcff8487053e7608e6 WatchSource:0}: Error finding container a5cdcfbf87e92b788d6f97f2cb192594f03da06048eb1fbcff8487053e7608e6: Status 404 returned error can't find the container with id a5cdcfbf87e92b788d6f97f2cb192594f03da06048eb1fbcff8487053e7608e6
Sep 14 02:50:55.428502 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.428485    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kklhq"
Sep 14 02:50:55.435873 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:55.435857    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ffab345_9789_435a_b2ae_b53ec8e66f43.slice/crio-7fde4b4ec82387b771f42ca67c600fe98bc569563312d3864131e37cb94c393a WatchSource:0}: Error finding container 7fde4b4ec82387b771f42ca67c600fe98bc569563312d3864131e37cb94c393a: Status 404 returned error can't find the container with id 7fde4b4ec82387b771f42ca67c600fe98bc569563312d3864131e37cb94c393a
Sep 14 02:50:55.439835 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.439810    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vfx28"
Sep 14 02:50:55.448097 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:55.448079    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60ad1800_10ba_4d33_a609_cb0e66dfdac5.slice/crio-0de43e893086bdcef9ac63a2c39b33dcd1c63407b0994b13ac05762d6e995012 WatchSource:0}: Error finding container 0de43e893086bdcef9ac63a2c39b33dcd1c63407b0994b13ac05762d6e995012: Status 404 returned error can't find the container with id 0de43e893086bdcef9ac63a2c39b33dcd1c63407b0994b13ac05762d6e995012
Sep 14 02:50:55.451999 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.451982    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5mpdv"
Sep 14 02:50:55.457249 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.457146    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nsgwq"
Sep 14 02:50:55.459182 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:55.459163    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c40ebe0_ae4f_407d_8f34_c1d4d4b46d45.slice/crio-947938f3d24e02c25450ec588a28c2e556f1792c1728119e06bbaa9c357f37ea WatchSource:0}: Error finding container 947938f3d24e02c25450ec588a28c2e556f1792c1728119e06bbaa9c357f37ea: Status 404 returned error can't find the container with id 947938f3d24e02c25450ec588a28c2e556f1792c1728119e06bbaa9c357f37ea
Sep 14 02:50:55.465341 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:55.465323    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a9bb9b8_0a23_445a_9074_930df7d80ea0.slice/crio-5f1d6c9518f0fda0eb7a5f3aa8b596c93c5395d31f2e81f8f7548e4109f28dad WatchSource:0}: Error finding container 5f1d6c9518f0fda0eb7a5f3aa8b596c93c5395d31f2e81f8f7548e4109f28dad: Status 404 returned error can't find the container with id 5f1d6c9518f0fda0eb7a5f3aa8b596c93c5395d31f2e81f8f7548e4109f28dad
Sep 14 02:50:55.467190 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.467126    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4lbmj"
Sep 14 02:50:55.472764 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.472746    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-84ls2"
Sep 14 02:50:55.481837 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:50:55.481818    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda851c347_abe6_4dce_8b64_5b40a3a812a8.slice/crio-a58a78e6cbb0670517b1e09dcdd3e94dad0c9411ab91a1f04b1ca0d43b614c35 WatchSource:0}: Error finding container a58a78e6cbb0670517b1e09dcdd3e94dad0c9411ab91a1f04b1ca0d43b614c35: Status 404 returned error can't find the container with id a58a78e6cbb0670517b1e09dcdd3e94dad0c9411ab91a1f04b1ca0d43b614c35
Sep 14 02:50:55.484617 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.484586    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-54ht7"
Sep 14 02:50:55.563460 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.563439    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 02:50:55.566894 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.566876    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 02:50:55.696013 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.695942    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs\") pod \"network-metrics-daemon-hkpjq\" (UID: \"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6\") " pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:50:55.696154 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.696025    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:50:55.696154 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.696122    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:55.696154 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.696145    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:55.696312 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.696194    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs podName:ae964ae5-7b42-42c5-8b19-1bf113cfd6d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:50:56.696173995 +0000 UTC m=+3.009197807 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs") pod "network-metrics-daemon-hkpjq" (UID: "ae964ae5-7b42-42c5-8b19-1bf113cfd6d6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:55.696312 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.696214    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret podName:2a5808dc-86c4-4507-8281-09e0c2910f77 nodeName:}" failed. No retries permitted until 2026-09-14 02:50:56.696204554 +0000 UTC m=+3.009228355 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret") pod "global-pull-secret-syncer-wdp4n" (UID: "2a5808dc-86c4-4507-8281-09e0c2910f77") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:55.897192 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:55.897157    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb95b\" (UniqueName: \"kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b\") pod \"network-check-target-5jzxs\" (UID: \"1d1732d2-7663-4622-bc8e-92f5c6d6555a\") " pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:50:55.897376 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.897331    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:50:55.897376 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.897356    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:50:55.897376 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.897370    2574 projected.go:196] Error preparing data for projected volume kube-api-access-qb95b for pod openshift-network-diagnostics/network-check-target-5jzxs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:55.897544 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:55.897433    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b podName:1d1732d2-7663-4622-bc8e-92f5c6d6555a nodeName:}" failed. No retries permitted until 2026-09-14 02:50:56.897411487 +0000 UTC m=+3.210435348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qb95b" (UniqueName: "kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b") pod "network-check-target-5jzxs" (UID: "1d1732d2-7663-4622-bc8e-92f5c6d6555a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:56.170955 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.170909    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 02:45:55 +0000 UTC" deadline="2028-06-17 11:34:53.013109668 +0000 UTC"
Sep 14 02:50:56.170955 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.170950    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15416h43m56.842164445s"
Sep 14 02:50:56.212978 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.212801    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:50:56.212978 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:56.212943    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:50:56.233045 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.232991    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-84ls2" event={"ID":"a851c347-abe6-4dce-8b64-5b40a3a812a8","Type":"ContainerStarted","Data":"a58a78e6cbb0670517b1e09dcdd3e94dad0c9411ab91a1f04b1ca0d43b614c35"}
Sep 14 02:50:56.249583 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.249544    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4lbmj" event={"ID":"d96e1f18-c6a1-47ff-96c5-a5954b361ff9","Type":"ContainerStarted","Data":"de4f7aa51b5e835a7a5a01a29552d859cd9ae2be1b10b41d174d33bb698e052b"}
Sep 14 02:50:56.267118 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.267083    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kklhq" event={"ID":"1ffab345-9789-435a-b2ae-b53ec8e66f43","Type":"ContainerStarted","Data":"7fde4b4ec82387b771f42ca67c600fe98bc569563312d3864131e37cb94c393a"}
Sep 14 02:50:56.282619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.282432    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62" event={"ID":"bc4edffb-356b-461d-ad74-52b9e26e243d","Type":"ContainerStarted","Data":"c1145752e71572c9e7699bc63767b9275966b29ab90c1b54881b0b92c4b7757e"}
Sep 14 02:50:56.295682 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.295649    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal" event={"ID":"07e0d660232ee4077dd70b8b3442d37b","Type":"ContainerStarted","Data":"0764b37a3bd59a69942d76db1a42b1ae30c27337fe4c84e6f0f67c2a057e572a"}
Sep 14 02:50:56.306178 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.306150    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nsgwq" event={"ID":"1a9bb9b8-0a23-445a-9074-930df7d80ea0","Type":"ContainerStarted","Data":"5f1d6c9518f0fda0eb7a5f3aa8b596c93c5395d31f2e81f8f7548e4109f28dad"}
Sep 14 02:50:56.312595 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.312522    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5mpdv" event={"ID":"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45","Type":"ContainerStarted","Data":"947938f3d24e02c25450ec588a28c2e556f1792c1728119e06bbaa9c357f37ea"}
Sep 14 02:50:56.338338 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.338311    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vfx28" event={"ID":"60ad1800-10ba-4d33-a609-cb0e66dfdac5","Type":"ContainerStarted","Data":"0de43e893086bdcef9ac63a2c39b33dcd1c63407b0994b13ac05762d6e995012"}
Sep 14 02:50:56.348463 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.348432    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp" event={"ID":"6608dc19-84cb-4438-8940-38f32c25381c","Type":"ContainerStarted","Data":"a5cdcfbf87e92b788d6f97f2cb192594f03da06048eb1fbcff8487053e7608e6"}
Sep 14 02:50:56.381362 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.381330    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal" event={"ID":"ab003f2b739da2d03a150c7e3dc691ab","Type":"ContainerStarted","Data":"6fecaab152855ecff4cbc0ac1ad3a7ccbf38f41d525a8572d3f81919e1454a4b"}
Sep 14 02:50:56.421245 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.421162    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54ht7" event={"ID":"b970a991-aaae-40da-9abb-4a84d3d53b61","Type":"ContainerStarted","Data":"7655f5f5593af630b26266b59b8faa44f6386d62b981063b20a18a3503f19da6"}
Sep 14 02:50:56.705274 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.705180    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:50:56.705274 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.705235    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs\") pod \"network-metrics-daemon-hkpjq\" (UID: \"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6\") " pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:50:56.705518 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:56.705403    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:56.705518 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:56.705464    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs podName:ae964ae5-7b42-42c5-8b19-1bf113cfd6d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:50:58.705443839 +0000 UTC m=+5.018467645 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs") pod "network-metrics-daemon-hkpjq" (UID: "ae964ae5-7b42-42c5-8b19-1bf113cfd6d6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:56.705705 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:56.705675    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:56.705775 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:56.705715    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret podName:2a5808dc-86c4-4507-8281-09e0c2910f77 nodeName:}" failed. No retries permitted until 2026-09-14 02:50:58.705703143 +0000 UTC m=+5.018726942 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret") pod "global-pull-secret-syncer-wdp4n" (UID: "2a5808dc-86c4-4507-8281-09e0c2910f77") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:56.907659 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:56.906899    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb95b\" (UniqueName: \"kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b\") pod \"network-check-target-5jzxs\" (UID: \"1d1732d2-7663-4622-bc8e-92f5c6d6555a\") " pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:50:56.907659 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:56.907076    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:50:56.907659 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:56.907099    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:50:56.907659 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:56.907112    2574 projected.go:196] Error preparing data for projected volume kube-api-access-qb95b for pod openshift-network-diagnostics/network-check-target-5jzxs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:56.907659 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:56.907179    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b podName:1d1732d2-7663-4622-bc8e-92f5c6d6555a nodeName:}" failed. No retries permitted until 2026-09-14 02:50:58.90715981 +0000 UTC m=+5.220183632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qb95b" (UniqueName: "kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b") pod "network-check-target-5jzxs" (UID: "1d1732d2-7663-4622-bc8e-92f5c6d6555a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:57.177176 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:57.176848    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 02:45:55 +0000 UTC" deadline="2028-03-27 22:45:52.436604235 +0000 UTC"
Sep 14 02:50:57.177176 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:57.176891    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13459h54m55.259718471s"
Sep 14 02:50:57.210559 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:57.209807    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:50:57.210559 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:57.209943    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:50:57.210559 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:57.210384    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:50:57.210559 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:57.210498    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:50:58.217991 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:58.213641    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:50:58.217991 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:58.213770    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:50:58.727995 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:58.727951    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:50:58.728187 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:58.728016    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs\") pod \"network-metrics-daemon-hkpjq\" (UID: \"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6\") " pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:50:58.728272 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:58.728190    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:58.728272 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:58.728262    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs podName:ae964ae5-7b42-42c5-8b19-1bf113cfd6d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:02.728241322 +0000 UTC m=+9.041265121 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs") pod "network-metrics-daemon-hkpjq" (UID: "ae964ae5-7b42-42c5-8b19-1bf113cfd6d6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:58.728735 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:58.728709    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:58.728859 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:58.728781    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret podName:2a5808dc-86c4-4507-8281-09e0c2910f77 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:02.728760475 +0000 UTC m=+9.041784290 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret") pod "global-pull-secret-syncer-wdp4n" (UID: "2a5808dc-86c4-4507-8281-09e0c2910f77") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:58.929941 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:58.929893    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb95b\" (UniqueName: \"kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b\") pod \"network-check-target-5jzxs\" (UID: \"1d1732d2-7663-4622-bc8e-92f5c6d6555a\") " pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:50:58.930124 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:58.930101    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:50:58.930177 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:58.930133    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:50:58.930177 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:58.930147    2574 projected.go:196] Error preparing data for projected volume kube-api-access-qb95b for pod openshift-network-diagnostics/network-check-target-5jzxs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:58.930253 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:58.930211    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b podName:1d1732d2-7663-4622-bc8e-92f5c6d6555a nodeName:}" failed. No retries permitted until 2026-09-14 02:51:02.930190428 +0000 UTC m=+9.243214233 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qb95b" (UniqueName: "kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b") pod "network-check-target-5jzxs" (UID: "1d1732d2-7663-4622-bc8e-92f5c6d6555a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:59.210058 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:59.209975    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:50:59.210217 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:59.210113    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:50:59.210533 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:50:59.210513    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:50:59.210668 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:50:59.210646    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:00.212659 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:00.212628    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:00.213043 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:00.212766    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:01.210659 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:01.209839    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:01.210659 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:01.210022    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:51:01.210659 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:01.210457    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:01.210659 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:01.210572    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:02.210411 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:02.210374    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:02.210837 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:02.210541    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:02.765731 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:02.765695    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:02.765905 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:02.765748    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs\") pod \"network-metrics-daemon-hkpjq\" (UID: \"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6\") " pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:02.765975 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:02.765906    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:51:02.765975 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:02.765964    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs podName:ae964ae5-7b42-42c5-8b19-1bf113cfd6d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:10.765945243 +0000 UTC m=+17.078969054 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs") pod "network-metrics-daemon-hkpjq" (UID: "ae964ae5-7b42-42c5-8b19-1bf113cfd6d6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:51:02.766086 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:02.766041    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:51:02.766086 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:02.766074    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret podName:2a5808dc-86c4-4507-8281-09e0c2910f77 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:10.7660626 +0000 UTC m=+17.079086412 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret") pod "global-pull-secret-syncer-wdp4n" (UID: "2a5808dc-86c4-4507-8281-09e0c2910f77") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:51:02.967816 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:02.967785    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb95b\" (UniqueName: \"kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b\") pod \"network-check-target-5jzxs\" (UID: \"1d1732d2-7663-4622-bc8e-92f5c6d6555a\") " pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:02.967993 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:02.967940    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:51:02.967993 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:02.967955    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:51:02.967993 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:02.967967    2574 projected.go:196] Error preparing data for projected volume kube-api-access-qb95b for pod openshift-network-diagnostics/network-check-target-5jzxs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:51:02.968148 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:02.968010    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b podName:1d1732d2-7663-4622-bc8e-92f5c6d6555a nodeName:}" failed. No retries permitted until 2026-09-14 02:51:10.967998611 +0000 UTC m=+17.281022413 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qb95b" (UniqueName: "kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b") pod "network-check-target-5jzxs" (UID: "1d1732d2-7663-4622-bc8e-92f5c6d6555a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:51:03.209885 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:03.209856    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:03.210056 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:03.209860    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:03.210056 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:03.209976    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:51:03.210174 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:03.210078    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:04.210905 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:04.210586    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:04.210905 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:04.210665    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:05.210020 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:05.209990    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:05.210020 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:05.210013    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:05.210211 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:05.210087    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:51:05.210247 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:05.210229    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:06.209621 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:06.209591    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:06.210083 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:06.209686    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:07.209530 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:07.209483    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:07.209718 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:07.209487    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:07.209718 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:07.209632    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:07.210090 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:07.209716    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:51:08.209479 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:08.209442    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:08.209709 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:08.209582    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:09.210421 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:09.210386    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:09.210845 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:09.210491    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:51:09.210845 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:09.210561    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:09.210845 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:09.210668    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:10.210345 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:10.210316    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:10.210504 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:10.210416    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:10.825380 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:10.825358    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:10.825570 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:10.825389    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs\") pod \"network-metrics-daemon-hkpjq\" (UID: \"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6\") " pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:10.825570 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:10.825466    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:51:10.825570 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:10.825501    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:51:10.825570 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:10.825525    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs podName:ae964ae5-7b42-42c5-8b19-1bf113cfd6d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:26.82549711 +0000 UTC m=+33.138520907 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs") pod "network-metrics-daemon-hkpjq" (UID: "ae964ae5-7b42-42c5-8b19-1bf113cfd6d6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:51:10.825725 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:10.825585    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret podName:2a5808dc-86c4-4507-8281-09e0c2910f77 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:26.825567699 +0000 UTC m=+33.138591502 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret") pod "global-pull-secret-syncer-wdp4n" (UID: "2a5808dc-86c4-4507-8281-09e0c2910f77") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:51:11.027783 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:11.027714    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb95b\" (UniqueName: \"kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b\") pod \"network-check-target-5jzxs\" (UID: \"1d1732d2-7663-4622-bc8e-92f5c6d6555a\") " pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:11.027911 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:11.027847    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:51:11.027911 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:11.027863    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:51:11.027911 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:11.027872    2574 projected.go:196] Error preparing data for projected volume kube-api-access-qb95b for pod openshift-network-diagnostics/network-check-target-5jzxs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:51:11.028039 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:11.027935    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b podName:1d1732d2-7663-4622-bc8e-92f5c6d6555a nodeName:}" failed. No retries permitted until 2026-09-14 02:51:27.027923302 +0000 UTC m=+33.340947112 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qb95b" (UniqueName: "kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b") pod "network-check-target-5jzxs" (UID: "1d1732d2-7663-4622-bc8e-92f5c6d6555a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:51:11.209736 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:11.209707    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:11.209924 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:11.209706    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:11.209924 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:11.209814    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:11.209924 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:11.209893    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:51:12.210457 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:12.210422    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:12.210955 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:12.210554    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:13.209398 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:13.209366    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:13.209398 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:13.209387    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:13.209667 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:13.209479    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:51:13.209667 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:13.209611    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:14.210369 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:14.210338    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:14.210839 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:14.210489    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:15.209813 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.209569    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:15.209944 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.209820    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:15.209944 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:15.209924    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:51:15.210055 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:15.210014    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:15.493730 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.493704    2574 generic.go:356] "Generic (PLEG): container finished" podID="ab003f2b739da2d03a150c7e3dc691ab" containerID="ae671c2b903e5bca5c9c63c44ea6a58faf230ff038aaef5355ba31fce2ec8def" exitCode=0
Sep 14 02:51:15.494229 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.493780    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal" event={"ID":"ab003f2b739da2d03a150c7e3dc691ab","Type":"ContainerDied","Data":"ae671c2b903e5bca5c9c63c44ea6a58faf230ff038aaef5355ba31fce2ec8def"}
Sep 14 02:51:15.495271 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.495251    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54ht7" event={"ID":"b970a991-aaae-40da-9abb-4a84d3d53b61","Type":"ContainerStarted","Data":"a861c9d45722b4ca6cccfd332aef1397ada5619ef6cd4d26bffc4536d764246d"}
Sep 14 02:51:15.496938 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.496918    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-84ls2" event={"ID":"a851c347-abe6-4dce-8b64-5b40a3a812a8","Type":"ContainerStarted","Data":"8916a79c98116926f520bece0da77c671bb4c8ecba90537cab3a5f092196c2a0"}
Sep 14 02:51:15.498073 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.498054    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4lbmj" event={"ID":"d96e1f18-c6a1-47ff-96c5-a5954b361ff9","Type":"ContainerStarted","Data":"25caeae59800bfd79dd18f5b2e77fd58c00a4607b9562318d6b044812747501a"}
Sep 14 02:51:15.499392 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.499345    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62" event={"ID":"bc4edffb-356b-461d-ad74-52b9e26e243d","Type":"ContainerStarted","Data":"59335e09d2eba7a45301c22a267a8d0bd32080b89ac555d36d9c5899a9ca444d"}
Sep 14 02:51:15.500756 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.500733    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal" event={"ID":"07e0d660232ee4077dd70b8b3442d37b","Type":"ContainerStarted","Data":"8fafc724798c016685633d8f57aebae3eecdaaf8920d1fe57c0ae5b298a16f1c"}
Sep 14 02:51:15.502372 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.502353    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nsgwq" event={"ID":"1a9bb9b8-0a23-445a-9074-930df7d80ea0","Type":"ContainerStarted","Data":"c0d1649780f2e35f0624c1a8b2cfadff3b78608d0835937a5ae5df7dc65cefe7"}
Sep 14 02:51:15.503836 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.503817    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5mpdv" event={"ID":"4c40ebe0-ae4f-407d-8f34-c1d4d4b46d45","Type":"ContainerStarted","Data":"0d2c8b6548e737f4fab2ac3b6b0dcb3869ed17422ba71f812dfa65306fa6ced2"}
Sep 14 02:51:15.505231 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.505214    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vfx28" event={"ID":"60ad1800-10ba-4d33-a609-cb0e66dfdac5","Type":"ContainerStarted","Data":"9a4a7c6ab04a9f8ff33a997e9225b1bcd140ace8b8692eb20972be1e222b8157"}
Sep 14 02:51:15.507504 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.507487    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 02:51:15.507891 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.507872    2574 generic.go:356] "Generic (PLEG): container finished" podID="6608dc19-84cb-4438-8940-38f32c25381c" containerID="5f2fcad9894b29aa7a39b9a8b8e4cc297bfb66f3a579d3b44a44a3b188eeed6a" exitCode=1
Sep 14 02:51:15.507948 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.507900    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp" event={"ID":"6608dc19-84cb-4438-8940-38f32c25381c","Type":"ContainerStarted","Data":"0af39f5d20c5832ab15b6967c79ff29038490e7da3103b1a3c610a66753b71f2"}
Sep 14 02:51:15.507993 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.507959    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp" event={"ID":"6608dc19-84cb-4438-8940-38f32c25381c","Type":"ContainerStarted","Data":"58f659fab90c3c6f177369975058bb303b3d8f092233e0e5bf3f3d8a589a90cc"}
Sep 14 02:51:15.507993 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.507991    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp" event={"ID":"6608dc19-84cb-4438-8940-38f32c25381c","Type":"ContainerStarted","Data":"3f669c9c81abfa324a9a935ea76306477a952b5ae85dab0a59b4a2cfc53cbc22"}
Sep 14 02:51:15.508074 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.508026    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp" event={"ID":"6608dc19-84cb-4438-8940-38f32c25381c","Type":"ContainerDied","Data":"5f2fcad9894b29aa7a39b9a8b8e4cc297bfb66f3a579d3b44a44a3b188eeed6a"}
Sep 14 02:51:15.508124 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.508071    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp" event={"ID":"6608dc19-84cb-4438-8940-38f32c25381c","Type":"ContainerStarted","Data":"ca0043ba1475cbeadec71fc9e433e9d34f5d3272c1c5ad268aa92663b3cc7591"}
Sep 14 02:51:15.588660 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.588617    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nsgwq" podStartSLOduration=1.3546528740000001 podStartE2EDuration="20.588601066s" totalImagesPullingTime="19.233948192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:50:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:15.559557404 +0000 UTC m=+21.872581362" watchObservedRunningTime="2026-09-14 02:51:15.588601066 +0000 UTC m=+21.901624884"
Sep 14 02:51:15.608717 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.608678    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4lbmj" podStartSLOduration=1.360032876 podStartE2EDuration="20.608663986s" totalImagesPullingTime="19.24863111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:15.588694093 +0000 UTC m=+21.901717912" watchObservedRunningTime="2026-09-14 02:51:15.608663986 +0000 UTC m=+21.921687802"
Sep 14 02:51:15.608800 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.608766    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-84ls2" podStartSLOduration=1.390996886 podStartE2EDuration="20.608762685s" totalImagesPullingTime="19.217765799s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:15.608657853 +0000 UTC m=+21.921681683" watchObservedRunningTime="2026-09-14 02:51:15.608762685 +0000 UTC m=+21.921786533"
Sep 14 02:51:15.658445 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.658409    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-188.ec2.internal" podStartSLOduration=20.658388122 podStartE2EDuration="20.658388122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:15.630188842 +0000 UTC m=+21.943212665" watchObservedRunningTime="2026-09-14 02:51:15.658388122 +0000 UTC m=+21.971411941"
Sep 14 02:51:15.658712 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.658683    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vfx28" podStartSLOduration=1.3835207729999999 podStartE2EDuration="20.658665291s" totalImagesPullingTime="19.275144518s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:15.658629334 +0000 UTC m=+21.971653147" watchObservedRunningTime="2026-09-14 02:51:15.658665291 +0000 UTC m=+21.971689109"
Sep 14 02:51:15.692115 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:15.692084    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5mpdv" podStartSLOduration=1.167793747 podStartE2EDuration="20.692071995s" totalImagesPullingTime="19.524278248s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:15.691592257 +0000 UTC m=+22.004616076" watchObservedRunningTime="2026-09-14 02:51:15.692071995 +0000 UTC m=+22.005095815"
Sep 14 02:51:16.209788 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:16.209759    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:16.209913 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:16.209885    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:16.512814 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:16.512793    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 02:51:16.513346 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:16.513193    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp" event={"ID":"6608dc19-84cb-4438-8940-38f32c25381c","Type":"ContainerStarted","Data":"ae2af7136ebabe2720a8b5279ae1ad4ba6d3d7fd2e2680e8ce4a265f93c89e0f"}
Sep 14 02:51:16.514499 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:16.514477    2574 generic.go:356] "Generic (PLEG): container finished" podID="b970a991-aaae-40da-9abb-4a84d3d53b61" containerID="a861c9d45722b4ca6cccfd332aef1397ada5619ef6cd4d26bffc4536d764246d" exitCode=0
Sep 14 02:51:16.514603 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:16.514553    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54ht7" event={"ID":"b970a991-aaae-40da-9abb-4a84d3d53b61","Type":"ContainerDied","Data":"a861c9d45722b4ca6cccfd332aef1397ada5619ef6cd4d26bffc4536d764246d"}
Sep 14 02:51:16.515951 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:16.515930    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kklhq" event={"ID":"1ffab345-9789-435a-b2ae-b53ec8e66f43","Type":"ContainerStarted","Data":"0e6cbfe6990d73240ee6cfd297a05df49293eaac34ea1ae0b20a129ca07d2a51"}
Sep 14 02:51:16.579860 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:16.579628    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kklhq" podStartSLOduration=2.31594136 podStartE2EDuration="21.579608472s" totalImagesPullingTime="19.263667112s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:16.579482691 +0000 UTC m=+22.892506511" watchObservedRunningTime="2026-09-14 02:51:16.579608472 +0000 UTC m=+22.892632292"
Sep 14 02:51:16.681268 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:16.680932    2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 14 02:51:17.165418 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:17.165324    2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-14T02:51:16.680951234Z","UUID":"eba6d8d0-ff1a-4bca-9fb3-6a3f9ad05089","Handler":null,"Name":"","Endpoint":""}
Sep 14 02:51:17.167110 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:17.167087    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 14 02:51:17.167233 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:17.167121    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 14 02:51:17.209619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:17.209595    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:17.209619 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:17.209602    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:17.209773 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:17.209718    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:51:17.209836 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:17.209815    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:17.520860 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:17.520816    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62" event={"ID":"bc4edffb-356b-461d-ad74-52b9e26e243d","Type":"ContainerStarted","Data":"d0676c6ac9a7d6c5c9daff757253328dedd7dc1b1c35febee091dd30dade32f6"}
Sep 14 02:51:17.523055 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:17.523022    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal" event={"ID":"ab003f2b739da2d03a150c7e3dc691ab","Type":"ContainerStarted","Data":"b095cc8f7434708f4a96ce22b8e19d839d7dc56ee60aff21059a6686ef9e5c34"}
Sep 14 02:51:17.545306 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:17.545249    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-188.ec2.internal" podStartSLOduration=22.545229775 podStartE2EDuration="22.545229775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:17.544952389 +0000 UTC m=+23.857976208" watchObservedRunningTime="2026-09-14 02:51:17.545229775 +0000 UTC m=+23.858253596"
Sep 14 02:51:18.213393 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:18.213320    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:18.213548 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:18.213426    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:18.528433 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:18.528391    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62" event={"ID":"bc4edffb-356b-461d-ad74-52b9e26e243d","Type":"ContainerStarted","Data":"cd02223b85c9f4720c90a5b20ec0b047d88ae141fcffc20d69121bc181b65dd4"}
Sep 14 02:51:18.532217 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:18.532198    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 02:51:18.532653 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:18.532626    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp" event={"ID":"6608dc19-84cb-4438-8940-38f32c25381c","Type":"ContainerStarted","Data":"994739ed23260e84cd617ee4acf3bbb4389b754c459750ccceb906efab19945c"}
Sep 14 02:51:19.209634 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:19.209595    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:19.209820 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:19.209712    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:51:19.209820 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:19.209770    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:19.209925 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:19.209883    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:20.066817 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:20.066728    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nsgwq"
Sep 14 02:51:20.067507 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:20.067487    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nsgwq"
Sep 14 02:51:20.085163 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:20.085110    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhj62" podStartSLOduration=4.448783898 podStartE2EDuration="26.085093242s" totalImagesPullingTime="21.636309344s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:18.56308172 +0000 UTC m=+24.876105539" watchObservedRunningTime="2026-09-14 02:51:20.085093242 +0000 UTC m=+26.398117086"
Sep 14 02:51:20.209996 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:20.209965    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:20.210144 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:20.210091    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:20.537364 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:20.537338    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nsgwq"
Sep 14 02:51:20.537948 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:20.537928    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nsgwq"
Sep 14 02:51:21.209496 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:21.209336    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:21.209892 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:21.209367    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:21.209892 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:21.209620    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:21.209892 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:21.209648    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:51:21.542109 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:21.542085    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 02:51:21.542490 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:21.542464    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp" event={"ID":"6608dc19-84cb-4438-8940-38f32c25381c","Type":"ContainerStarted","Data":"b3e726b9f733b58568016ff77b89f6b5699e1ca1d183cde211fd2c74f3c50b2b"}
Sep 14 02:51:21.542831 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:21.542808    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:51:21.542922 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:21.542887    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:51:21.542974 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:21.542920    2574 scope.go:117] "RemoveContainer" containerID="5f2fcad9894b29aa7a39b9a8b8e4cc297bfb66f3a579d3b44a44a3b188eeed6a"
Sep 14 02:51:21.542974 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:21.542937    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:51:21.544960 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:21.544934    2574 generic.go:356] "Generic (PLEG): container finished" podID="b970a991-aaae-40da-9abb-4a84d3d53b61" containerID="f611c8e199309b34835610502cd4b8e4a73ec15a5011174d63902947856b7ceb" exitCode=0
Sep 14 02:51:21.545069 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:21.545041    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54ht7" event={"ID":"b970a991-aaae-40da-9abb-4a84d3d53b61","Type":"ContainerDied","Data":"f611c8e199309b34835610502cd4b8e4a73ec15a5011174d63902947856b7ceb"}
Sep 14 02:51:21.560425 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:21.560407    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:51:21.561693 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:21.561673    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:51:22.209748 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:22.209718    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:22.210368 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:22.209827    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:22.552005 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:22.551982    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 02:51:22.552364 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:22.552341    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp" event={"ID":"6608dc19-84cb-4438-8940-38f32c25381c","Type":"ContainerStarted","Data":"93315f77e97ff5d122e8f840b2936382567b988198b056a9d936174fe0f31f5d"}
Sep 14 02:51:22.844561 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:22.844304    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp" podStartSLOduration=9.422697626 podStartE2EDuration="28.844287203s" totalImagesPullingTime="19.421589577s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:22.60827608 +0000 UTC m=+28.921299883" watchObservedRunningTime="2026-09-14 02:51:22.844287203 +0000 UTC m=+29.157311021"
Sep 14 02:51:22.844950 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:22.844931    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wdp4n"]
Sep 14 02:51:22.845086 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:22.845069    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:22.845172 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:22.845153    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:51:22.857798 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:22.857773    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hkpjq"]
Sep 14 02:51:22.857896 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:22.857884    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:22.858012 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:22.857993    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:22.862738 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:22.862716    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5jzxs"]
Sep 14 02:51:22.862807 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:22.862796    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:22.862872 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:22.862856    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:23.556912 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:23.556882    2574 generic.go:356] "Generic (PLEG): container finished" podID="b970a991-aaae-40da-9abb-4a84d3d53b61" containerID="842da5dddb8e04db2894b010b013216d3eae60655c01393d77bc70e9c9455920" exitCode=0
Sep 14 02:51:23.557391 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:23.556973    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54ht7" event={"ID":"b970a991-aaae-40da-9abb-4a84d3d53b61","Type":"ContainerDied","Data":"842da5dddb8e04db2894b010b013216d3eae60655c01393d77bc70e9c9455920"}
Sep 14 02:51:24.210365 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:24.210304    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:24.210506 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:24.210395    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:51:24.210506 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:24.210468    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:24.210610 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:24.210583    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:24.210643 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:24.210624    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:24.210720 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:24.210702    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:25.564099 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:25.564061    2574 generic.go:356] "Generic (PLEG): container finished" podID="b970a991-aaae-40da-9abb-4a84d3d53b61" containerID="656cca22c9bdd2357a2b5d8b72c0458151d94a17a42048e5455c56259436a8df" exitCode=0
Sep 14 02:51:25.564576 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:25.564146    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54ht7" event={"ID":"b970a991-aaae-40da-9abb-4a84d3d53b61","Type":"ContainerDied","Data":"656cca22c9bdd2357a2b5d8b72c0458151d94a17a42048e5455c56259436a8df"}
Sep 14 02:51:26.209652 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:26.209622    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:26.209837 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:26.209622    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:26.209837 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:26.209732    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:26.209944 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:26.209622    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:26.209944 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:26.209835    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:51:26.209944 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:26.209937    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:26.845763 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:26.845732    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:26.846182 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:26.845775    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs\") pod \"network-metrics-daemon-hkpjq\" (UID: \"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6\") " pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:26.846182 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:26.845892    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:51:26.846182 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:26.845891    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:51:26.846182 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:26.845959    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs podName:ae964ae5-7b42-42c5-8b19-1bf113cfd6d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:58.845940516 +0000 UTC m=+65.158964327 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs") pod "network-metrics-daemon-hkpjq" (UID: "ae964ae5-7b42-42c5-8b19-1bf113cfd6d6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:51:26.846182 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:26.846021    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret podName:2a5808dc-86c4-4507-8281-09e0c2910f77 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:58.846001772 +0000 UTC m=+65.159025582 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret") pod "global-pull-secret-syncer-wdp4n" (UID: "2a5808dc-86c4-4507-8281-09e0c2910f77") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:51:27.048087 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:27.048050    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb95b\" (UniqueName: \"kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b\") pod \"network-check-target-5jzxs\" (UID: \"1d1732d2-7663-4622-bc8e-92f5c6d6555a\") " pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:27.048254 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:27.048218    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:51:27.048254 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:27.048242    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:51:27.048254 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:27.048256    2574 projected.go:196] Error preparing data for projected volume kube-api-access-qb95b for pod openshift-network-diagnostics/network-check-target-5jzxs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:51:27.048398 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:27.048319    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b podName:1d1732d2-7663-4622-bc8e-92f5c6d6555a nodeName:}" failed. No retries permitted until 2026-09-14 02:51:59.048299977 +0000 UTC m=+65.361323787 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qb95b" (UniqueName: "kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b") pod "network-check-target-5jzxs" (UID: "1d1732d2-7663-4622-bc8e-92f5c6d6555a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:51:28.209641 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:28.209559    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:28.209641 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:28.209591    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:28.209641 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:28.209563    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:28.210284 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:28.209703    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hkpjq" podUID="ae964ae5-7b42-42c5-8b19-1bf113cfd6d6"
Sep 14 02:51:28.210284 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:28.209809    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wdp4n" podUID="2a5808dc-86c4-4507-8281-09e0c2910f77"
Sep 14 02:51:28.210284 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:28.209897    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5jzxs" podUID="1d1732d2-7663-4622-bc8e-92f5c6d6555a"
Sep 14 02:51:28.944818 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:28.944748    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-188.ec2.internal" event="NodeReady"
Sep 14 02:51:28.944973 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:28.944876    2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 14 02:51:29.037525 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.037482    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7dxw9"]
Sep 14 02:51:29.042958 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.042939    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7dxw9"
Sep 14 02:51:29.045113 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.045089    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kxpfw"]
Sep 14 02:51:29.045735 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.045713    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hnrbf\""
Sep 14 02:51:29.046035 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.046014    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 14 02:51:29.046380 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.046362    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 14 02:51:29.046467 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.046407    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 14 02:51:29.049029 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.048774    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:29.053673 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.053655    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 14 02:51:29.053781 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.053761    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nzbvq\""
Sep 14 02:51:29.053891 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.053800    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 14 02:51:29.064322 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.064303    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7dxw9"]
Sep 14 02:51:29.088888 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.088865    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kxpfw"]
Sep 14 02:51:29.164862 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.164835    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t569x\" (UniqueName: \"kubernetes.io/projected/d85ac4b3-b0a0-42d7-a99d-8170602d4170-kube-api-access-t569x\") pod \"ingress-canary-7dxw9\" (UID: \"d85ac4b3-b0a0-42d7-a99d-8170602d4170\") " pod="openshift-ingress-canary/ingress-canary-7dxw9"
Sep 14 02:51:29.165005 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.164881    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a144e24-363d-48b1-a4a0-b27d07c109ba-tmp-dir\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:29.165005 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.164921    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxhsr\" (UniqueName: \"kubernetes.io/projected/6a144e24-363d-48b1-a4a0-b27d07c109ba-kube-api-access-zxhsr\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:29.165005 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.164988    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d85ac4b3-b0a0-42d7-a99d-8170602d4170-cert\") pod \"ingress-canary-7dxw9\" (UID: \"d85ac4b3-b0a0-42d7-a99d-8170602d4170\") " pod="openshift-ingress-canary/ingress-canary-7dxw9"
Sep 14 02:51:29.165138 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.165009    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a144e24-363d-48b1-a4a0-b27d07c109ba-config-volume\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:29.165138 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.165037    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:29.265423 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.265395    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a144e24-363d-48b1-a4a0-b27d07c109ba-tmp-dir\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:29.265849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.265457    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxhsr\" (UniqueName: \"kubernetes.io/projected/6a144e24-363d-48b1-a4a0-b27d07c109ba-kube-api-access-zxhsr\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:29.265849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.265478    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d85ac4b3-b0a0-42d7-a99d-8170602d4170-cert\") pod \"ingress-canary-7dxw9\" (UID: \"d85ac4b3-b0a0-42d7-a99d-8170602d4170\") " pod="openshift-ingress-canary/ingress-canary-7dxw9"
Sep 14 02:51:29.265849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.265494    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a144e24-363d-48b1-a4a0-b27d07c109ba-config-volume\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:29.265849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.265527    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:29.265849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.265567    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t569x\" (UniqueName: \"kubernetes.io/projected/d85ac4b3-b0a0-42d7-a99d-8170602d4170-kube-api-access-t569x\") pod \"ingress-canary-7dxw9\" (UID: \"d85ac4b3-b0a0-42d7-a99d-8170602d4170\") " pod="openshift-ingress-canary/ingress-canary-7dxw9"
Sep 14 02:51:29.265849 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:29.265632    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:51:29.265849 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:29.265697    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls podName:6a144e24-363d-48b1-a4a0-b27d07c109ba nodeName:}" failed. No retries permitted until 2026-09-14 02:51:29.765676444 +0000 UTC m=+36.078700240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls") pod "dns-default-kxpfw" (UID: "6a144e24-363d-48b1-a4a0-b27d07c109ba") : secret "dns-default-metrics-tls" not found
Sep 14 02:51:29.265849 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.265832    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a144e24-363d-48b1-a4a0-b27d07c109ba-tmp-dir\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:29.266271 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.266175    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a144e24-363d-48b1-a4a0-b27d07c109ba-config-volume\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:29.270547 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.270390    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d85ac4b3-b0a0-42d7-a99d-8170602d4170-cert\") pod \"ingress-canary-7dxw9\" (UID: \"d85ac4b3-b0a0-42d7-a99d-8170602d4170\") " pod="openshift-ingress-canary/ingress-canary-7dxw9"
Sep 14 02:51:29.276870 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.276848    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxhsr\" (UniqueName: \"kubernetes.io/projected/6a144e24-363d-48b1-a4a0-b27d07c109ba-kube-api-access-zxhsr\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:29.277842 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.277820    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t569x\" (UniqueName: \"kubernetes.io/projected/d85ac4b3-b0a0-42d7-a99d-8170602d4170-kube-api-access-t569x\") pod \"ingress-canary-7dxw9\" (UID: \"d85ac4b3-b0a0-42d7-a99d-8170602d4170\") " pod="openshift-ingress-canary/ingress-canary-7dxw9"
Sep 14 02:51:29.355205 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.355178    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7dxw9"
Sep 14 02:51:29.762628 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.762606    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7dxw9"]
Sep 14 02:51:29.770004 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:29.769983    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:29.770182 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:29.770160    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:51:29.770254 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:29.770243    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls podName:6a144e24-363d-48b1-a4a0-b27d07c109ba nodeName:}" failed. No retries permitted until 2026-09-14 02:51:30.770222921 +0000 UTC m=+37.083246721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls") pod "dns-default-kxpfw" (UID: "6a144e24-363d-48b1-a4a0-b27d07c109ba") : secret "dns-default-metrics-tls" not found
Sep 14 02:51:29.792342 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:51:29.792314    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd85ac4b3_b0a0_42d7_a99d_8170602d4170.slice/crio-dee246fd58959a26a2251fcd425d55c76944d9bc5d09d4a93f705487086c5c22 WatchSource:0}: Error finding container dee246fd58959a26a2251fcd425d55c76944d9bc5d09d4a93f705487086c5c22: Status 404 returned error can't find the container with id dee246fd58959a26a2251fcd425d55c76944d9bc5d09d4a93f705487086c5c22
Sep 14 02:51:30.209992 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:30.209916    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:30.210133 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:30.209916    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:30.210374 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:30.209925    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:30.215115 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:30.215095    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 02:51:30.215225 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:30.215152    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mk9cr\""
Sep 14 02:51:30.215291 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:30.215097    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 02:51:30.215889 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:30.215871    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 02:51:30.216025 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:30.215908    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b88wl\""
Sep 14 02:51:30.216025 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:30.215928    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 02:51:30.578375 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:30.578335    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7dxw9" event={"ID":"d85ac4b3-b0a0-42d7-a99d-8170602d4170","Type":"ContainerStarted","Data":"dee246fd58959a26a2251fcd425d55c76944d9bc5d09d4a93f705487086c5c22"}
Sep 14 02:51:30.581706 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:30.581675    2574 generic.go:356] "Generic (PLEG): container finished" podID="b970a991-aaae-40da-9abb-4a84d3d53b61" containerID="5ce9bda2b9391b1cf6025b71abb3bb3ad90f7e17c70d51b8ba02418770fd0520" exitCode=0
Sep 14 02:51:30.581837 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:30.581729    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54ht7" event={"ID":"b970a991-aaae-40da-9abb-4a84d3d53b61","Type":"ContainerDied","Data":"5ce9bda2b9391b1cf6025b71abb3bb3ad90f7e17c70d51b8ba02418770fd0520"}
Sep 14 02:51:30.778043 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:30.778002    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:30.778222 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:30.778207    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:51:30.778290 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:30.778258    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls podName:6a144e24-363d-48b1-a4a0-b27d07c109ba nodeName:}" failed. No retries permitted until 2026-09-14 02:51:32.778244618 +0000 UTC m=+39.091268415 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls") pod "dns-default-kxpfw" (UID: "6a144e24-363d-48b1-a4a0-b27d07c109ba") : secret "dns-default-metrics-tls" not found
Sep 14 02:51:31.588030 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:31.587997    2574 generic.go:356] "Generic (PLEG): container finished" podID="b970a991-aaae-40da-9abb-4a84d3d53b61" containerID="eb280c287bccfa5d653e7ca91fd6e687028610dc89bd21c1b607e8944f83e5f6" exitCode=0
Sep 14 02:51:31.588620 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:31.588060    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54ht7" event={"ID":"b970a991-aaae-40da-9abb-4a84d3d53b61","Type":"ContainerDied","Data":"eb280c287bccfa5d653e7ca91fd6e687028610dc89bd21c1b607e8944f83e5f6"}
Sep 14 02:51:32.593247 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:32.592907    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7dxw9" event={"ID":"d85ac4b3-b0a0-42d7-a99d-8170602d4170","Type":"ContainerStarted","Data":"19929535f332dd7b79896a6b258e2684307c6ca56839f03c8928b7ed75834e43"}
Sep 14 02:51:32.596960 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:32.596929    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54ht7" event={"ID":"b970a991-aaae-40da-9abb-4a84d3d53b61","Type":"ContainerStarted","Data":"ceac7bfdb0ed369193c4efa3c08050314209ecb70c6807e4c4a92f9f5aed4837"}
Sep 14 02:51:32.620853 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:32.620808    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7dxw9" podStartSLOduration=1.072004143 podStartE2EDuration="3.620793269s" totalImagesPullingTime="2.548789126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:51:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:32.620219345 +0000 UTC m=+38.933243165" watchObservedRunningTime="2026-09-14 02:51:32.620793269 +0000 UTC m=+38.933817082"
Sep 14 02:51:32.656878 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:32.656833    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-54ht7" podStartSLOduration=8.009345348 podStartE2EDuration="37.656816644s" totalImagesPullingTime="29.647471296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:32.654438706 +0000 UTC m=+38.967462525" watchObservedRunningTime="2026-09-14 02:51:32.656816644 +0000 UTC m=+38.969840463"
Sep 14 02:51:32.793181 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:32.793158    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:32.793296 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:32.793239    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:51:32.793296 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:32.793293    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls podName:6a144e24-363d-48b1-a4a0-b27d07c109ba nodeName:}" failed. No retries permitted until 2026-09-14 02:51:36.793280364 +0000 UTC m=+43.106304161 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls") pod "dns-default-kxpfw" (UID: "6a144e24-363d-48b1-a4a0-b27d07c109ba") : secret "dns-default-metrics-tls" not found
Sep 14 02:51:36.815732 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:36.815703    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:36.816081 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:36.815824    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:51:36.816081 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:36.815877    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls podName:6a144e24-363d-48b1-a4a0-b27d07c109ba nodeName:}" failed. No retries permitted until 2026-09-14 02:51:44.815861616 +0000 UTC m=+51.128885434 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls") pod "dns-default-kxpfw" (UID: "6a144e24-363d-48b1-a4a0-b27d07c109ba") : secret "dns-default-metrics-tls" not found
Sep 14 02:51:44.872532 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:44.872475    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:51:44.872933 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:44.872599    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:51:44.872933 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:44.872652    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls podName:6a144e24-363d-48b1-a4a0-b27d07c109ba nodeName:}" failed. No retries permitted until 2026-09-14 02:52:00.872637886 +0000 UTC m=+67.185661682 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls") pod "dns-default-kxpfw" (UID: "6a144e24-363d-48b1-a4a0-b27d07c109ba") : secret "dns-default-metrics-tls" not found
Sep 14 02:51:53.578809 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:53.578774    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4jxp"
Sep 14 02:51:58.743600 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.743567    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-mrh5b"]
Sep 14 02:51:58.745857 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.745842    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-mrh5b"
Sep 14 02:51:58.748415 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.748394    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 14 02:51:58.748503 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.748467    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8s4dp\""
Sep 14 02:51:58.748608 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.748592    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 14 02:51:58.756413 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.756395    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-lnp2z"]
Sep 14 02:51:58.757915 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.757897    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-b66mm"]
Sep 14 02:51:58.758023 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.758009    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:51:58.759705 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.759675    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-mrh5b"]
Sep 14 02:51:58.759818 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.759764    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-b66mm"
Sep 14 02:51:58.760625 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.760603    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 14 02:51:58.760832 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.760630    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 14 02:51:58.760832 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.760647    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 14 02:51:58.760832 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.760668    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-xqbx6\""
Sep 14 02:51:58.761018 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.761006    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 14 02:51:58.762395 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.762377    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qxmsg\""
Sep 14 02:51:58.763883 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.763865    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8e2a325e-9309-46ae-8fca-2cafce3df418-trusted-ca\") pod \"console-operator-696884fb9c-lnp2z\" (UID: \"8e2a325e-9309-46ae-8fca-2cafce3df418\") " pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:51:58.763969 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.763889    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnbj5\" (UniqueName: \"kubernetes.io/projected/8e2a325e-9309-46ae-8fca-2cafce3df418-kube-api-access-pnbj5\") pod \"console-operator-696884fb9c-lnp2z\" (UID: \"8e2a325e-9309-46ae-8fca-2cafce3df418\") " pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:51:58.763969 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.763906    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e2a325e-9309-46ae-8fca-2cafce3df418-serving-cert\") pod \"console-operator-696884fb9c-lnp2z\" (UID: \"8e2a325e-9309-46ae-8fca-2cafce3df418\") " pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:51:58.763969 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.763939    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtblx\" (UniqueName: \"kubernetes.io/projected/1ffb8f86-fa93-4852-a5f2-3154414af556-kube-api-access-gtblx\") pod \"network-check-source-6f8b768455-b66mm\" (UID: \"1ffb8f86-fa93-4852-a5f2-3154414af556\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-b66mm"
Sep 14 02:51:58.764118 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.763991    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp5fp\" (UniqueName: \"kubernetes.io/projected/8e237d4c-dde5-4191-84f6-e8e32ff7d5d8-kube-api-access-rp5fp\") pod \"volume-data-source-validator-86695b7cc7-mrh5b\" (UID: \"8e237d4c-dde5-4191-84f6-e8e32ff7d5d8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-mrh5b"
Sep 14 02:51:58.764118 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.764058    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e2a325e-9309-46ae-8fca-2cafce3df418-config\") pod \"console-operator-696884fb9c-lnp2z\" (UID: \"8e2a325e-9309-46ae-8fca-2cafce3df418\") " pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:51:58.765584 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.765566    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 14 02:51:58.769235 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.769217    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-b66mm"]
Sep 14 02:51:58.772579 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.772405    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-lnp2z"]
Sep 14 02:51:58.855480 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.855455    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-6gxb8"]
Sep 14 02:51:58.857038 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.857024    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.859062 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.859032    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"]
Sep 14 02:51:58.861114 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.860563    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 14 02:51:58.861114 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.860707    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 14 02:51:58.861114 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.860817    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 14 02:51:58.861114 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.861101    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"
Sep 14 02:51:58.861401 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.861366    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 14 02:51:58.862182 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.861922    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-czbx6\""
Sep 14 02:51:58.865053 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.864845    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae97ab3e-852d-4891-84bd-579e977e7aee-service-ca-bundle\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.865053 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.864898    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8e2a325e-9309-46ae-8fca-2cafce3df418-trusted-ca\") pod \"console-operator-696884fb9c-lnp2z\" (UID: \"8e2a325e-9309-46ae-8fca-2cafce3df418\") " pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:51:58.865053 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.864930    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnbj5\" (UniqueName: \"kubernetes.io/projected/8e2a325e-9309-46ae-8fca-2cafce3df418-kube-api-access-pnbj5\") pod \"console-operator-696884fb9c-lnp2z\" (UID: \"8e2a325e-9309-46ae-8fca-2cafce3df418\") " pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:51:58.865053 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.864954    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae97ab3e-852d-4891-84bd-579e977e7aee-tmp\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.865053 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.864978    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e2a325e-9309-46ae-8fca-2cafce3df418-serving-cert\") pod \"console-operator-696884fb9c-lnp2z\" (UID: \"8e2a325e-9309-46ae-8fca-2cafce3df418\") " pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:51:58.865053 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.865000    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-m929g\" (UID: \"41133492-26f4-43c9-907d-94694d564d69\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"
Sep 14 02:51:58.865053 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.865025    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ae97ab3e-852d-4891-84bd-579e977e7aee-serving-cert\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.865053 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.865052    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9jvc\" (UniqueName: \"kubernetes.io/projected/ae97ab3e-852d-4891-84bd-579e977e7aee-kube-api-access-n9jvc\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.865473 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.865073    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 14 02:51:58.865473 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.865082    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 14 02:51:58.865473 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.865076    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtblx\" (UniqueName: \"kubernetes.io/projected/1ffb8f86-fa93-4852-a5f2-3154414af556-kube-api-access-gtblx\") pod \"network-check-source-6f8b768455-b66mm\" (UID: \"1ffb8f86-fa93-4852-a5f2-3154414af556\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-b66mm"
Sep 14 02:51:58.865473 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.865300    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae97ab3e-852d-4891-84bd-579e977e7aee-trusted-ca-bundle\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.865473 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.865341    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:58.865473 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.865378    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 14 02:51:58.865950 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.865375    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6rdv\" (UniqueName: \"kubernetes.io/projected/41133492-26f4-43c9-907d-94694d564d69-kube-api-access-p6rdv\") pod \"cluster-samples-operator-86499c89c4-m929g\" (UID: \"41133492-26f4-43c9-907d-94694d564d69\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"
Sep 14 02:51:58.865950 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.865580    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs\") pod \"network-metrics-daemon-hkpjq\" (UID: \"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6\") " pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:51:58.865950 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.865666    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp5fp\" (UniqueName: \"kubernetes.io/projected/8e237d4c-dde5-4191-84f6-e8e32ff7d5d8-kube-api-access-rp5fp\") pod \"volume-data-source-validator-86695b7cc7-mrh5b\" (UID: \"8e237d4c-dde5-4191-84f6-e8e32ff7d5d8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-mrh5b"
Sep 14 02:51:58.865950 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.865731    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7ntvs\""
Sep 14 02:51:58.865950 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.865732    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ae97ab3e-852d-4891-84bd-579e977e7aee-snapshots\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.865950 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.865811    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e2a325e-9309-46ae-8fca-2cafce3df418-config\") pod \"console-operator-696884fb9c-lnp2z\" (UID: \"8e2a325e-9309-46ae-8fca-2cafce3df418\") " pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:51:58.866249 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.866071    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8e2a325e-9309-46ae-8fca-2cafce3df418-trusted-ca\") pod \"console-operator-696884fb9c-lnp2z\" (UID: \"8e2a325e-9309-46ae-8fca-2cafce3df418\") " pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:51:58.866547 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.866505    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e2a325e-9309-46ae-8fca-2cafce3df418-config\") pod \"console-operator-696884fb9c-lnp2z\" (UID: \"8e2a325e-9309-46ae-8fca-2cafce3df418\") " pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:51:58.868560 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.868541    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 02:51:58.868666 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.868587    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"]
Sep 14 02:51:58.869336 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.869315    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e2a325e-9309-46ae-8fca-2cafce3df418-serving-cert\") pod \"console-operator-696884fb9c-lnp2z\" (UID: \"8e2a325e-9309-46ae-8fca-2cafce3df418\") " pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:51:58.869595 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.869580    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 14 02:51:58.869870 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.869853    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 02:51:58.870159 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.870143    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:51:58.872385 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.872368    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-6gxb8"]
Sep 14 02:51:58.873565 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:58.873537    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Sep 14 02:51:58.873666 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:58.873599    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-monitoring-operator-tls\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" type="*v1.Secret"
Sep 14 02:51:58.873666 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:58.873605    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 14 02:51:58.873791 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:58.873696    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"telemetry-config\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" type="*v1.ConfigMap"
Sep 14 02:51:58.874456 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.874433    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-m57nm\""
Sep 14 02:51:58.874615 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.874591    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"]
Sep 14 02:51:58.876045 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:58.876026    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 14 02:51:58.876137 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:58.876082    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs podName:ae964ae5-7b42-42c5-8b19-1bf113cfd6d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:02.876065305 +0000 UTC m=+129.189089107 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs") pod "network-metrics-daemon-hkpjq" (UID: "ae964ae5-7b42-42c5-8b19-1bf113cfd6d6") : secret "metrics-daemon-secret" not found
Sep 14 02:51:58.877966 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.877936    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a5808dc-86c4-4507-8281-09e0c2910f77-original-pull-secret\") pod \"global-pull-secret-syncer-wdp4n\" (UID: \"2a5808dc-86c4-4507-8281-09e0c2910f77\") " pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:58.880202 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.880181    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp5fp\" (UniqueName: \"kubernetes.io/projected/8e237d4c-dde5-4191-84f6-e8e32ff7d5d8-kube-api-access-rp5fp\") pod \"volume-data-source-validator-86695b7cc7-mrh5b\" (UID: \"8e237d4c-dde5-4191-84f6-e8e32ff7d5d8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-mrh5b"
Sep 14 02:51:58.884847 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.884829    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"]
Sep 14 02:51:58.887432 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.887412    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnbj5\" (UniqueName: \"kubernetes.io/projected/8e2a325e-9309-46ae-8fca-2cafce3df418-kube-api-access-pnbj5\") pod \"console-operator-696884fb9c-lnp2z\" (UID: \"8e2a325e-9309-46ae-8fca-2cafce3df418\") " pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:51:58.887636 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.887618    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 02:51:58.898599 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.898583    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 02:51:58.907840 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.907821    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtblx\" (UniqueName: \"kubernetes.io/projected/1ffb8f86-fa93-4852-a5f2-3154414af556-kube-api-access-gtblx\") pod \"network-check-source-6f8b768455-b66mm\" (UID: \"1ffb8f86-fa93-4852-a5f2-3154414af556\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-b66mm"
Sep 14 02:51:58.956975 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.956954    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr"]
Sep 14 02:51:58.958827 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.958812    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr"
Sep 14 02:51:58.961287 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.961271    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 14 02:51:58.961577 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.961564    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-h66w4\""
Sep 14 02:51:58.961822 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.961809    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 14 02:51:58.961863 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.961834    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 14 02:51:58.962159 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.962139    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 14 02:51:58.966120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966101    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae97ab3e-852d-4891-84bd-579e977e7aee-service-ca-bundle\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.966271 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966135    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-m9bnr\" (UID: \"c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr"
Sep 14 02:51:58.966271 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966177    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-fkpqf\" (UID: \"ff4eb2f8-e574-4e2c-a59f-0c715f466adb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:51:58.966271 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966203    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae97ab3e-852d-4891-84bd-579e977e7aee-tmp\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.966271 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966228    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-m929g\" (UID: \"41133492-26f4-43c9-907d-94694d564d69\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"
Sep 14 02:51:58.966471 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966277    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ae97ab3e-852d-4891-84bd-579e977e7aee-serving-cert\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.966471 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966307    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9jvc\" (UniqueName: \"kubernetes.io/projected/ae97ab3e-852d-4891-84bd-579e977e7aee-kube-api-access-n9jvc\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.966471 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:58.966322    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 02:51:58.966471 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966328    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fkpqf\" (UID: \"ff4eb2f8-e574-4e2c-a59f-0c715f466adb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:51:58.966471 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966357    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae97ab3e-852d-4891-84bd-579e977e7aee-trusted-ca-bundle\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.966471 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:58.966373    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls podName:41133492-26f4-43c9-907d-94694d564d69 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:59.466356769 +0000 UTC m=+65.779380569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-m929g" (UID: "41133492-26f4-43c9-907d-94694d564d69") : secret "samples-operator-tls" not found
Sep 14 02:51:58.966471 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966405    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmt86\" (UniqueName: \"kubernetes.io/projected/c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b-kube-api-access-gmt86\") pod \"kube-storage-version-migrator-operator-6c8f84f897-m9bnr\" (UID: \"c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr"
Sep 14 02:51:58.966471 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966458    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj7pq\" (UniqueName: \"kubernetes.io/projected/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-kube-api-access-pj7pq\") pod \"cluster-monitoring-operator-6cd954b897-fkpqf\" (UID: \"ff4eb2f8-e574-4e2c-a59f-0c715f466adb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:51:58.966873 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966499    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6rdv\" (UniqueName: \"kubernetes.io/projected/41133492-26f4-43c9-907d-94694d564d69-kube-api-access-p6rdv\") pod \"cluster-samples-operator-86499c89c4-m929g\" (UID: \"41133492-26f4-43c9-907d-94694d564d69\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"
Sep 14 02:51:58.966873 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966557    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae97ab3e-852d-4891-84bd-579e977e7aee-tmp\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.966873 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966558    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-m9bnr\" (UID: \"c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr"
Sep 14 02:51:58.966873 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966619    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ae97ab3e-852d-4891-84bd-579e977e7aee-snapshots\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.966873 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.966772    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae97ab3e-852d-4891-84bd-579e977e7aee-service-ca-bundle\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.967052 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.967010    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ae97ab3e-852d-4891-84bd-579e977e7aee-snapshots\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.967085 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.967060    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae97ab3e-852d-4891-84bd-579e977e7aee-trusted-ca-bundle\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.968412 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.968394    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ae97ab3e-852d-4891-84bd-579e977e7aee-serving-cert\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.972403 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.972387    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr"]
Sep 14 02:51:58.981240 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.981217    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9jvc\" (UniqueName: \"kubernetes.io/projected/ae97ab3e-852d-4891-84bd-579e977e7aee-kube-api-access-n9jvc\") pod \"insights-operator-5844b856db-6gxb8\" (UID: \"ae97ab3e-852d-4891-84bd-579e977e7aee\") " pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:58.984591 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:58.984573    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6rdv\" (UniqueName: \"kubernetes.io/projected/41133492-26f4-43c9-907d-94694d564d69-kube-api-access-p6rdv\") pod \"cluster-samples-operator-86499c89c4-m929g\" (UID: \"41133492-26f4-43c9-907d-94694d564d69\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"
Sep 14 02:51:59.034455 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.034409    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wdp4n"
Sep 14 02:51:59.054605 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.054583    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-mrh5b"
Sep 14 02:51:59.066980 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.066952    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb95b\" (UniqueName: \"kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b\") pod \"network-check-target-5jzxs\" (UID: \"1d1732d2-7663-4622-bc8e-92f5c6d6555a\") " pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:59.067048 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.067009    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-m9bnr\" (UID: \"c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr"
Sep 14 02:51:59.067093 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.067052    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-fkpqf\" (UID: \"ff4eb2f8-e574-4e2c-a59f-0c715f466adb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:51:59.067189 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.067172    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fkpqf\" (UID: \"ff4eb2f8-e574-4e2c-a59f-0c715f466adb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:51:59.067235 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.067208    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmt86\" (UniqueName: \"kubernetes.io/projected/c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b-kube-api-access-gmt86\") pod \"kube-storage-version-migrator-operator-6c8f84f897-m9bnr\" (UID: \"c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr"
Sep 14 02:51:59.067283 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.067239    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj7pq\" (UniqueName: \"kubernetes.io/projected/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-kube-api-access-pj7pq\") pod \"cluster-monitoring-operator-6cd954b897-fkpqf\" (UID: \"ff4eb2f8-e574-4e2c-a59f-0c715f466adb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:51:59.067433 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.067409    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-m9bnr\" (UID: \"c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr"
Sep 14 02:51:59.068096 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.068062    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-m9bnr\" (UID: \"c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr"
Sep 14 02:51:59.069728 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.069709    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:51:59.070093 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.070070    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-m9bnr\" (UID: \"c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr"
Sep 14 02:51:59.070341 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.070328    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb95b\" (UniqueName: \"kubernetes.io/projected/1d1732d2-7663-4622-bc8e-92f5c6d6555a-kube-api-access-qb95b\") pod \"network-check-target-5jzxs\" (UID: \"1d1732d2-7663-4622-bc8e-92f5c6d6555a\") " pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:59.075274 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.074680    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-b66mm"
Sep 14 02:51:59.076238 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.076205    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmt86\" (UniqueName: \"kubernetes.io/projected/c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b-kube-api-access-gmt86\") pod \"kube-storage-version-migrator-operator-6c8f84f897-m9bnr\" (UID: \"c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr"
Sep 14 02:51:59.170690 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.170655    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-6gxb8"
Sep 14 02:51:59.266252 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.266217    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr"
Sep 14 02:51:59.305971 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.305942    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wdp4n"]
Sep 14 02:51:59.308416 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:51:59.308389    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a5808dc_86c4_4507_8281_09e0c2910f77.slice/crio-20dd90acf6bb51d7acfd565189c657b5a750b2659660bc3a10be6b886042ba83 WatchSource:0}: Error finding container 20dd90acf6bb51d7acfd565189c657b5a750b2659660bc3a10be6b886042ba83: Status 404 returned error can't find the container with id 20dd90acf6bb51d7acfd565189c657b5a750b2659660bc3a10be6b886042ba83
Sep 14 02:51:59.325949 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.325930    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b88wl\""
Sep 14 02:51:59.334247 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.334227    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:51:59.343228 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.343200    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-mrh5b"]
Sep 14 02:51:59.346308 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:51:59.346288    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e237d4c_dde5_4191_84f6_e8e32ff7d5d8.slice/crio-0853506fd6051f0d1484074f8dac513f95a0a2d5e60b6b4b2423fee13b9e5327 WatchSource:0}: Error finding container 0853506fd6051f0d1484074f8dac513f95a0a2d5e60b6b4b2423fee13b9e5327: Status 404 returned error can't find the container with id 0853506fd6051f0d1484074f8dac513f95a0a2d5e60b6b4b2423fee13b9e5327
Sep 14 02:51:59.361199 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.361176    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-b66mm"]
Sep 14 02:51:59.363611 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.363592    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-lnp2z"]
Sep 14 02:51:59.364562 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:51:59.364530    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ffb8f86_fa93_4852_a5f2_3154414af556.slice/crio-4da2549aedeefabfd0019f8bd7800818ba4650cebef2b6b27a0b72f14ef0eee8 WatchSource:0}: Error finding container 4da2549aedeefabfd0019f8bd7800818ba4650cebef2b6b27a0b72f14ef0eee8: Status 404 returned error can't find the container with id 4da2549aedeefabfd0019f8bd7800818ba4650cebef2b6b27a0b72f14ef0eee8
Sep 14 02:51:59.365576 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:51:59.365559    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e2a325e_9309_46ae_8fca_2cafce3df418.slice/crio-18ff287bbe24dd2b99405c5af7943f4d9f86abb5157c7024b5eb03eb5a90c7e6 WatchSource:0}: Error finding container 18ff287bbe24dd2b99405c5af7943f4d9f86abb5157c7024b5eb03eb5a90c7e6: Status 404 returned error can't find the container with id 18ff287bbe24dd2b99405c5af7943f4d9f86abb5157c7024b5eb03eb5a90c7e6
Sep 14 02:51:59.435916 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.435845    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-6gxb8"]
Sep 14 02:51:59.437691 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:51:59.437663    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae97ab3e_852d_4891_84bd_579e977e7aee.slice/crio-0740aa3a36ca374a72e4a49a9ae971b24d8477230899ed3ab5fe06e19d16aa42 WatchSource:0}: Error finding container 0740aa3a36ca374a72e4a49a9ae971b24d8477230899ed3ab5fe06e19d16aa42: Status 404 returned error can't find the container with id 0740aa3a36ca374a72e4a49a9ae971b24d8477230899ed3ab5fe06e19d16aa42
Sep 14 02:51:59.469662 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.469643    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-m929g\" (UID: \"41133492-26f4-43c9-907d-94694d564d69\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"
Sep 14 02:51:59.469812 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:59.469793    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 02:51:59.469861 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:59.469847    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls podName:41133492-26f4-43c9-907d-94694d564d69 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:00.469833152 +0000 UTC m=+66.782856952 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-m929g" (UID: "41133492-26f4-43c9-907d-94694d564d69") : secret "samples-operator-tls" not found
Sep 14 02:51:59.530434 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.530414    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr"]
Sep 14 02:51:59.532810 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:51:59.532789    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2e2f2db_8ee5_45e5_b800_5c7b26aa3b7b.slice/crio-f93a11f8b84dac4b4361efc228d17926051ab0f583932336412597e968c004ed WatchSource:0}: Error finding container f93a11f8b84dac4b4361efc228d17926051ab0f583932336412597e968c004ed: Status 404 returned error can't find the container with id f93a11f8b84dac4b4361efc228d17926051ab0f583932336412597e968c004ed
Sep 14 02:51:59.609179 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.609113    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5jzxs"]
Sep 14 02:51:59.612227 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:51:59.612198    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d1732d2_7663_4622_bc8e_92f5c6d6555a.slice/crio-8a3fad58ec5e7fa43342a09a27da702d4d050d98b21b0bb4a19fc9a9810087ef WatchSource:0}: Error finding container 8a3fad58ec5e7fa43342a09a27da702d4d050d98b21b0bb4a19fc9a9810087ef: Status 404 returned error can't find the container with id 8a3fad58ec5e7fa43342a09a27da702d4d050d98b21b0bb4a19fc9a9810087ef
Sep 14 02:51:59.663782 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.663749    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5jzxs" event={"ID":"1d1732d2-7663-4622-bc8e-92f5c6d6555a","Type":"ContainerStarted","Data":"8a3fad58ec5e7fa43342a09a27da702d4d050d98b21b0bb4a19fc9a9810087ef"}
Sep 14 02:51:59.664856 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.664831    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wdp4n" event={"ID":"2a5808dc-86c4-4507-8281-09e0c2910f77","Type":"ContainerStarted","Data":"20dd90acf6bb51d7acfd565189c657b5a750b2659660bc3a10be6b886042ba83"}
Sep 14 02:51:59.665993 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.665961    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr" event={"ID":"c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b","Type":"ContainerStarted","Data":"f93a11f8b84dac4b4361efc228d17926051ab0f583932336412597e968c004ed"}
Sep 14 02:51:59.667179 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.667157    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-b66mm" event={"ID":"1ffb8f86-fa93-4852-a5f2-3154414af556","Type":"ContainerStarted","Data":"4da2549aedeefabfd0019f8bd7800818ba4650cebef2b6b27a0b72f14ef0eee8"}
Sep 14 02:51:59.668221 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.668200    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-lnp2z" event={"ID":"8e2a325e-9309-46ae-8fca-2cafce3df418","Type":"ContainerStarted","Data":"18ff287bbe24dd2b99405c5af7943f4d9f86abb5157c7024b5eb03eb5a90c7e6"}
Sep 14 02:51:59.669299 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.669279    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-6gxb8" event={"ID":"ae97ab3e-852d-4891-84bd-579e977e7aee","Type":"ContainerStarted","Data":"0740aa3a36ca374a72e4a49a9ae971b24d8477230899ed3ab5fe06e19d16aa42"}
Sep 14 02:51:59.670275 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.670258    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-mrh5b" event={"ID":"8e237d4c-dde5-4191-84f6-e8e32ff7d5d8","Type":"ContainerStarted","Data":"0853506fd6051f0d1484074f8dac513f95a0a2d5e60b6b4b2423fee13b9e5327"}
Sep 14 02:51:59.730955 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.730930    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 14 02:51:59.845692 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.845667    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 14 02:51:59.848128 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.848107    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-fkpqf\" (UID: \"ff4eb2f8-e574-4e2c-a59f-0c715f466adb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:51:59.901445 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.901227    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 14 02:51:59.907532 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.907259    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj7pq\" (UniqueName: \"kubernetes.io/projected/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-kube-api-access-pj7pq\") pod \"cluster-monitoring-operator-6cd954b897-fkpqf\" (UID: \"ff4eb2f8-e574-4e2c-a59f-0c715f466adb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:51:59.978455 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:51:59.978432    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 14 02:51:59.988165 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:59.987919    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 02:51:59.988165 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:51:59.987985    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls podName:ff4eb2f8-e574-4e2c-a59f-0c715f466adb nodeName:}" failed. No retries permitted until 2026-09-14 02:52:00.487965392 +0000 UTC m=+66.800989202 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-fkpqf" (UID: "ff4eb2f8-e574-4e2c-a59f-0c715f466adb") : secret "cluster-monitoring-operator-tls" not found
Sep 14 02:52:00.479525 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:00.479471    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-m929g\" (UID: \"41133492-26f4-43c9-907d-94694d564d69\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"
Sep 14 02:52:00.479701 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:00.479655    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 02:52:00.479767 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:00.479725    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls podName:41133492-26f4-43c9-907d-94694d564d69 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:02.479705483 +0000 UTC m=+68.792729303 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-m929g" (UID: "41133492-26f4-43c9-907d-94694d564d69") : secret "samples-operator-tls" not found
Sep 14 02:52:00.580816 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:00.580769    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fkpqf\" (UID: \"ff4eb2f8-e574-4e2c-a59f-0c715f466adb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:52:00.581003 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:00.580979    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 02:52:00.581067 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:00.581046    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls podName:ff4eb2f8-e574-4e2c-a59f-0c715f466adb nodeName:}" failed. No retries permitted until 2026-09-14 02:52:01.581028315 +0000 UTC m=+67.894052127 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-fkpqf" (UID: "ff4eb2f8-e574-4e2c-a59f-0c715f466adb") : secret "cluster-monitoring-operator-tls" not found
Sep 14 02:52:00.883762 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:00.883679    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:52:00.884326 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:00.883857    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:52:00.884326 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:00.883917    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls podName:6a144e24-363d-48b1-a4a0-b27d07c109ba nodeName:}" failed. No retries permitted until 2026-09-14 02:52:32.883899979 +0000 UTC m=+99.196923795 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls") pod "dns-default-kxpfw" (UID: "6a144e24-363d-48b1-a4a0-b27d07c109ba") : secret "dns-default-metrics-tls" not found
Sep 14 02:52:01.590779 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:01.590685    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fkpqf\" (UID: \"ff4eb2f8-e574-4e2c-a59f-0c715f466adb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:52:01.590999 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:01.590981    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 02:52:01.591078 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:01.591048    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls podName:ff4eb2f8-e574-4e2c-a59f-0c715f466adb nodeName:}" failed. No retries permitted until 2026-09-14 02:52:03.591027469 +0000 UTC m=+69.904051280 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-fkpqf" (UID: "ff4eb2f8-e574-4e2c-a59f-0c715f466adb") : secret "cluster-monitoring-operator-tls" not found
Sep 14 02:52:01.686773 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:01.686736    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-mrh5b" event={"ID":"8e237d4c-dde5-4191-84f6-e8e32ff7d5d8","Type":"ContainerStarted","Data":"5e06972354a17844fec863890eee433d329c4092c85f84faa3edfb5e9d9ae8b9"}
Sep 14 02:52:01.708267 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:01.707827    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-mrh5b" podStartSLOduration=1.7127591949999998 podStartE2EDuration="3.707788598s" totalImagesPullingTime="1.995029403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:01.706872903 +0000 UTC m=+68.019896723" watchObservedRunningTime="2026-09-14 02:52:01.707788598 +0000 UTC m=+68.020812416"
Sep 14 02:52:02.499180 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:02.499088    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-m929g\" (UID: \"41133492-26f4-43c9-907d-94694d564d69\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"
Sep 14 02:52:02.499688 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:02.499256    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 02:52:02.499688 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:02.499328    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls podName:41133492-26f4-43c9-907d-94694d564d69 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:06.499308792 +0000 UTC m=+72.812332594 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-m929g" (UID: "41133492-26f4-43c9-907d-94694d564d69") : secret "samples-operator-tls" not found
Sep 14 02:52:03.608727 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:03.608685    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fkpqf\" (UID: \"ff4eb2f8-e574-4e2c-a59f-0c715f466adb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:52:03.609180 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:03.608875    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 02:52:03.609180 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:03.608941    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls podName:ff4eb2f8-e574-4e2c-a59f-0c715f466adb nodeName:}" failed. No retries permitted until 2026-09-14 02:52:07.608922416 +0000 UTC m=+73.921946225 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-fkpqf" (UID: "ff4eb2f8-e574-4e2c-a59f-0c715f466adb") : secret "cluster-monitoring-operator-tls" not found
Sep 14 02:52:06.532339 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.532310    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-m929g\" (UID: \"41133492-26f4-43c9-907d-94694d564d69\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"
Sep 14 02:52:06.532697 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:06.532444    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 02:52:06.532697 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:06.532509    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls podName:41133492-26f4-43c9-907d-94694d564d69 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:14.532489831 +0000 UTC m=+80.845513633 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-m929g" (UID: "41133492-26f4-43c9-907d-94694d564d69") : secret "samples-operator-tls" not found
Sep 14 02:52:06.712450 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.712418    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5jzxs" event={"ID":"1d1732d2-7663-4622-bc8e-92f5c6d6555a","Type":"ContainerStarted","Data":"8f5eaaf25293bf5301717fa6b9bd5e5f369ccee8ac53b8594124f2fd64e6a926"}
Sep 14 02:52:06.712641 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.712607    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:52:06.714233 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.714201    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wdp4n" event={"ID":"2a5808dc-86c4-4507-8281-09e0c2910f77","Type":"ContainerStarted","Data":"d3ef316d1a8778f8aebc927a24970befe05207013d2caa30e3c96e0f5b26408e"}
Sep 14 02:52:06.716247 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.715909    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr" event={"ID":"c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b","Type":"ContainerStarted","Data":"50f85c431a0fbf760409ecb3c6503116fb4ce64be2855af10a1f9a2e8b929040"}
Sep 14 02:52:06.717668 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.717647    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-b66mm" event={"ID":"1ffb8f86-fa93-4852-a5f2-3154414af556","Type":"ContainerStarted","Data":"d588038cf80b0d569bee0a0dd6eb84e8ff5090fcbb6737131dfd4e853de51569"}
Sep 14 02:52:06.719433 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.719409    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/0.log"
Sep 14 02:52:06.719538 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.719447    2574 generic.go:356] "Generic (PLEG): container finished" podID="8e2a325e-9309-46ae-8fca-2cafce3df418" containerID="1134534e0b200e8d8e03e57f85da0e90c41954e06f75d7a60654595b10f5e0d8" exitCode=255
Sep 14 02:52:06.719612 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.719553    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-lnp2z" event={"ID":"8e2a325e-9309-46ae-8fca-2cafce3df418","Type":"ContainerDied","Data":"1134534e0b200e8d8e03e57f85da0e90c41954e06f75d7a60654595b10f5e0d8"}
Sep 14 02:52:06.719701 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.719685    2574 scope.go:117] "RemoveContainer" containerID="1134534e0b200e8d8e03e57f85da0e90c41954e06f75d7a60654595b10f5e0d8"
Sep 14 02:52:06.723359 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.723339    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-6gxb8" event={"ID":"ae97ab3e-852d-4891-84bd-579e977e7aee","Type":"ContainerStarted","Data":"065bb31c674785384d0ba3de1cf5157d34c50d6f406b16895d24a64da7716158"}
Sep 14 02:52:06.737218 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.737174    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5jzxs" podStartSLOduration=65.259654902 podStartE2EDuration="1m11.737159123s" totalImagesPullingTime="6.477504221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:50:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:06.735049861 +0000 UTC m=+73.048073685" watchObservedRunningTime="2026-09-14 02:52:06.737159123 +0000 UTC m=+73.050182941"
Sep 14 02:52:06.784873 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.784727    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-b66mm" podStartSLOduration=2.073224283 podStartE2EDuration="8.784703706s" totalImagesPullingTime="6.711479423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:06.758113474 +0000 UTC m=+73.071137293" watchObservedRunningTime="2026-09-14 02:52:06.784703706 +0000 UTC m=+73.097727527"
Sep 14 02:52:06.785632 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.785595    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr" podStartSLOduration=2.242347335 podStartE2EDuration="8.785582425s" totalImagesPullingTime="6.54323509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:06.78436273 +0000 UTC m=+73.097386551" watchObservedRunningTime="2026-09-14 02:52:06.785582425 +0000 UTC m=+73.098606235"
Sep 14 02:52:06.808943 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.808888    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wdp4n" podStartSLOduration=65.037100631 podStartE2EDuration="1m11.808868829s" totalImagesPullingTime="6.771768198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:06.808312592 +0000 UTC m=+73.121336412" watchObservedRunningTime="2026-09-14 02:52:06.808868829 +0000 UTC m=+73.121892649"
Sep 14 02:52:06.864633 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:06.863757    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-6gxb8" podStartSLOduration=2.225412794 podStartE2EDuration="8.863733357s" totalImagesPullingTime="6.638320563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:06.838424896 +0000 UTC m=+73.151448715" watchObservedRunningTime="2026-09-14 02:52:06.863733357 +0000 UTC m=+73.176757181"
Sep 14 02:52:07.642421 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:07.642385    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fkpqf\" (UID: \"ff4eb2f8-e574-4e2c-a59f-0c715f466adb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:52:07.642784 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:07.642559    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 02:52:07.642784 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:07.642626    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls podName:ff4eb2f8-e574-4e2c-a59f-0c715f466adb nodeName:}" failed. No retries permitted until 2026-09-14 02:52:15.642610793 +0000 UTC m=+81.955634589 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-fkpqf" (UID: "ff4eb2f8-e574-4e2c-a59f-0c715f466adb") : secret "cluster-monitoring-operator-tls" not found
Sep 14 02:52:07.734660 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:07.734636    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 02:52:07.735087 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:07.735073    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/0.log"
Sep 14 02:52:07.735151 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:07.735106    2574 generic.go:356] "Generic (PLEG): container finished" podID="8e2a325e-9309-46ae-8fca-2cafce3df418" containerID="55e4d055d5e6247661e00942a0caf8bf63a0634bf715714771079cb5f610607c" exitCode=255
Sep 14 02:52:07.735208 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:07.735180    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-lnp2z" event={"ID":"8e2a325e-9309-46ae-8fca-2cafce3df418","Type":"ContainerDied","Data":"55e4d055d5e6247661e00942a0caf8bf63a0634bf715714771079cb5f610607c"}
Sep 14 02:52:07.735293 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:07.735281    2574 scope.go:117] "RemoveContainer" containerID="1134534e0b200e8d8e03e57f85da0e90c41954e06f75d7a60654595b10f5e0d8"
Sep 14 02:52:07.735507 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:07.735483    2574 scope.go:117] "RemoveContainer" containerID="55e4d055d5e6247661e00942a0caf8bf63a0634bf715714771079cb5f610607c"
Sep 14 02:52:07.735716 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:07.735688    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-lnp2z_openshift-console-operator(8e2a325e-9309-46ae-8fca-2cafce3df418)\"" pod="openshift-console-operator/console-operator-696884fb9c-lnp2z" podUID="8e2a325e-9309-46ae-8fca-2cafce3df418"
Sep 14 02:52:08.740581 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:08.740556    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 02:52:08.740944 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:08.740837    2574 scope.go:117] "RemoveContainer" containerID="55e4d055d5e6247661e00942a0caf8bf63a0634bf715714771079cb5f610607c"
Sep 14 02:52:08.740978 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:08.740966    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-lnp2z_openshift-console-operator(8e2a325e-9309-46ae-8fca-2cafce3df418)\"" pod="openshift-console-operator/console-operator-696884fb9c-lnp2z" podUID="8e2a325e-9309-46ae-8fca-2cafce3df418"
Sep 14 02:52:09.070816 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:09.070746    2574 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:52:09.070816 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:09.070797    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:52:09.744117 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:09.744091    2574 scope.go:117] "RemoveContainer" containerID="55e4d055d5e6247661e00942a0caf8bf63a0634bf715714771079cb5f610607c"
Sep 14 02:52:09.744482 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:09.744235    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-lnp2z_openshift-console-operator(8e2a325e-9309-46ae-8fca-2cafce3df418)\"" pod="openshift-console-operator/console-operator-696884fb9c-lnp2z" podUID="8e2a325e-9309-46ae-8fca-2cafce3df418"
Sep 14 02:52:10.383484 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:10.383458    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vfx28_60ad1800-10ba-4d33-a609-cb0e66dfdac5/dns-node-resolver/0.log"
Sep 14 02:52:11.183180 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:11.183144    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-84ls2_a851c347-abe6-4dce-8b64-5b40a3a812a8/node-ca/0.log"
Sep 14 02:52:11.985685 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:11.985656    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7dxw9_d85ac4b3-b0a0-42d7-a99d-8170602d4170/serve-healthcheck-canary/0.log"
Sep 14 02:52:12.585859 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:12.585828    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-m9bnr_c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b/kube-storage-version-migrator-operator/0.log"
Sep 14 02:52:14.589221 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:14.589188    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-m929g\" (UID: \"41133492-26f4-43c9-907d-94694d564d69\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"
Sep 14 02:52:14.591732 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:14.591712    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/41133492-26f4-43c9-907d-94694d564d69-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-m929g\" (UID: \"41133492-26f4-43c9-907d-94694d564d69\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"
Sep 14 02:52:14.796870 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:14.796843    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"
Sep 14 02:52:15.065075 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:15.065045    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g"]
Sep 14 02:52:15.697457 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:15.697426    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fkpqf\" (UID: \"ff4eb2f8-e574-4e2c-a59f-0c715f466adb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:52:15.697845 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:15.697612    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 02:52:15.697845 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:15.697707    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls podName:ff4eb2f8-e574-4e2c-a59f-0c715f466adb nodeName:}" failed. No retries permitted until 2026-09-14 02:52:31.697687262 +0000 UTC m=+98.010711073 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-fkpqf" (UID: "ff4eb2f8-e574-4e2c-a59f-0c715f466adb") : secret "cluster-monitoring-operator-tls" not found
Sep 14 02:52:15.764933 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:15.764895    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g" event={"ID":"41133492-26f4-43c9-907d-94694d564d69","Type":"ContainerStarted","Data":"6d19c14db1307cedd499b063bbb6c5a41d93c526a6542f5411752f16f8573439"}
Sep 14 02:52:17.774331 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:17.774285    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g" event={"ID":"41133492-26f4-43c9-907d-94694d564d69","Type":"ContainerStarted","Data":"101aa59a59abc70635c60f00ae35989a400396226386dbba62d1607b7d777d5e"}
Sep 14 02:52:17.774813 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:17.774340    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g" event={"ID":"41133492-26f4-43c9-907d-94694d564d69","Type":"ContainerStarted","Data":"1399bf4c749d49afcf7c66d03fcce4a4f64e42eea4246f289f1ce543a7028a8e"}
Sep 14 02:52:17.791596 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:17.791549    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-m929g" podStartSLOduration=18.028052012 podStartE2EDuration="19.791533759s" totalImagesPullingTime="1.763481747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:17.790786763 +0000 UTC m=+84.103810581" watchObservedRunningTime="2026-09-14 02:52:17.791533759 +0000 UTC m=+84.104557568"
Sep 14 02:52:24.211266 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:24.211237    2574 scope.go:117] "RemoveContainer" containerID="55e4d055d5e6247661e00942a0caf8bf63a0634bf715714771079cb5f610607c"
Sep 14 02:52:24.798268 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:24.798242    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 02:52:24.798430 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:24.798316    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-lnp2z" event={"ID":"8e2a325e-9309-46ae-8fca-2cafce3df418","Type":"ContainerStarted","Data":"8ba55b4dd408205951a8f1c11acdc063b61e5ea97dfe883abb112c553fa637c5"}
Sep 14 02:52:24.798607 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:24.798587    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:52:24.803325 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:24.803304    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-lnp2z"
Sep 14 02:52:24.816217 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:24.816170    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-lnp2z" podStartSLOduration=20.106117342 podStartE2EDuration="26.816155717s" totalImagesPullingTime="6.710038375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:24.815740687 +0000 UTC m=+91.128764507" watchObservedRunningTime="2026-09-14 02:52:24.816155717 +0000 UTC m=+91.129179537"
Sep 14 02:52:26.047041 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.047010    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-p95kw"]
Sep 14 02:52:26.050020 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.050005    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.054679 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.054660    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pmfgr\""
Sep 14 02:52:26.054811 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.054708    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 14 02:52:26.054811 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.054663    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 14 02:52:26.072089 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.072059    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p95kw"]
Sep 14 02:52:26.077792 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.077768    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"]
Sep 14 02:52:26.081032 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.081013    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.087481 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.087460    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 14 02:52:26.087612 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.087484    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 14 02:52:26.087612 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.087565    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 14 02:52:26.087723 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.087638    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 14 02:52:26.087927 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.087910    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 14 02:52:26.088023 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.087931    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 14 02:52:26.088508 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.088493    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 14 02:52:26.107835 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.107809    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"]
Sep 14 02:52:26.173806 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.173769    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/34720b3c-3761-4c43-acda-d281e9b35cdc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.173806 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.173810    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fedea9f9-1caf-464e-94e1-31ca4f616882-data-volume\") pod \"insights-runtime-extractor-p95kw\" (UID: \"fedea9f9-1caf-464e-94e1-31ca4f616882\") " pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.173999 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.173831    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fedea9f9-1caf-464e-94e1-31ca4f616882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p95kw\" (UID: \"fedea9f9-1caf-464e-94e1-31ca4f616882\") " pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.173999 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.173918    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fedea9f9-1caf-464e-94e1-31ca4f616882-crio-socket\") pod \"insights-runtime-extractor-p95kw\" (UID: \"fedea9f9-1caf-464e-94e1-31ca4f616882\") " pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.173999 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.173957    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/34720b3c-3761-4c43-acda-d281e9b35cdc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.173999 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.173975    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbt8k\" (UniqueName: \"kubernetes.io/projected/34720b3c-3761-4c43-acda-d281e9b35cdc-kube-api-access-wbt8k\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.174120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.174026    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/34720b3c-3761-4c43-acda-d281e9b35cdc-ca\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.174120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.174061    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68mb7\" (UniqueName: \"kubernetes.io/projected/fedea9f9-1caf-464e-94e1-31ca4f616882-kube-api-access-68mb7\") pod \"insights-runtime-extractor-p95kw\" (UID: \"fedea9f9-1caf-464e-94e1-31ca4f616882\") " pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.174120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.174084    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/34720b3c-3761-4c43-acda-d281e9b35cdc-hub\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.174120 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.174102    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fedea9f9-1caf-464e-94e1-31ca4f616882-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p95kw\" (UID: \"fedea9f9-1caf-464e-94e1-31ca4f616882\") " pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.174237 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.174123    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/34720b3c-3761-4c43-acda-d281e9b35cdc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.274953 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.274916    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/34720b3c-3761-4c43-acda-d281e9b35cdc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.274953 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.274945    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbt8k\" (UniqueName: \"kubernetes.io/projected/34720b3c-3761-4c43-acda-d281e9b35cdc-kube-api-access-wbt8k\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.275177 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.274976    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/34720b3c-3761-4c43-acda-d281e9b35cdc-ca\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.275177 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.275010    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68mb7\" (UniqueName: \"kubernetes.io/projected/fedea9f9-1caf-464e-94e1-31ca4f616882-kube-api-access-68mb7\") pod \"insights-runtime-extractor-p95kw\" (UID: \"fedea9f9-1caf-464e-94e1-31ca4f616882\") " pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.275177 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.275025    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/34720b3c-3761-4c43-acda-d281e9b35cdc-hub\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.275177 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.275047    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fedea9f9-1caf-464e-94e1-31ca4f616882-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p95kw\" (UID: \"fedea9f9-1caf-464e-94e1-31ca4f616882\") " pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.275177 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.275103    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/34720b3c-3761-4c43-acda-d281e9b35cdc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.275177 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.275169    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/34720b3c-3761-4c43-acda-d281e9b35cdc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.275564 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.275207    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fedea9f9-1caf-464e-94e1-31ca4f616882-data-volume\") pod \"insights-runtime-extractor-p95kw\" (UID: \"fedea9f9-1caf-464e-94e1-31ca4f616882\") " pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.275564 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.275242    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fedea9f9-1caf-464e-94e1-31ca4f616882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p95kw\" (UID: \"fedea9f9-1caf-464e-94e1-31ca4f616882\") " pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.275564 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.275299    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fedea9f9-1caf-464e-94e1-31ca4f616882-crio-socket\") pod \"insights-runtime-extractor-p95kw\" (UID: \"fedea9f9-1caf-464e-94e1-31ca4f616882\") " pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.275564 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.275383    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fedea9f9-1caf-464e-94e1-31ca4f616882-crio-socket\") pod \"insights-runtime-extractor-p95kw\" (UID: \"fedea9f9-1caf-464e-94e1-31ca4f616882\") " pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.275761 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.275703    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fedea9f9-1caf-464e-94e1-31ca4f616882-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p95kw\" (UID: \"fedea9f9-1caf-464e-94e1-31ca4f616882\") " pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.275817 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.275782    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/34720b3c-3761-4c43-acda-d281e9b35cdc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.276283 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.276262    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fedea9f9-1caf-464e-94e1-31ca4f616882-data-volume\") pod \"insights-runtime-extractor-p95kw\" (UID: \"fedea9f9-1caf-464e-94e1-31ca4f616882\") " pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.277841 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.277816    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fedea9f9-1caf-464e-94e1-31ca4f616882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p95kw\" (UID: \"fedea9f9-1caf-464e-94e1-31ca4f616882\") " pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.278181 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.278157    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/34720b3c-3761-4c43-acda-d281e9b35cdc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.278305 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.278289    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/34720b3c-3761-4c43-acda-d281e9b35cdc-hub\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.278384 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.278367    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/34720b3c-3761-4c43-acda-d281e9b35cdc-ca\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.278462 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.278448    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/34720b3c-3761-4c43-acda-d281e9b35cdc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.295078 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.295048    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68mb7\" (UniqueName: \"kubernetes.io/projected/fedea9f9-1caf-464e-94e1-31ca4f616882-kube-api-access-68mb7\") pod \"insights-runtime-extractor-p95kw\" (UID: \"fedea9f9-1caf-464e-94e1-31ca4f616882\") " pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.295463 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.295447    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbt8k\" (UniqueName: \"kubernetes.io/projected/34720b3c-3761-4c43-acda-d281e9b35cdc-kube-api-access-wbt8k\") pod \"cluster-proxy-proxy-agent-65dd976bc7-q7cf2\" (UID: \"34720b3c-3761-4c43-acda-d281e9b35cdc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.358993 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.358937    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p95kw"
Sep 14 02:52:26.399600 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.399571    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"
Sep 14 02:52:26.629350 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.629329    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p95kw"]
Sep 14 02:52:26.631487 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:52:26.631464    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfedea9f9_1caf_464e_94e1_31ca4f616882.slice/crio-cd6c46e02fa910044f9d1549abb7a3cd001c8285702dc6d90132e3f31795583a WatchSource:0}: Error finding container cd6c46e02fa910044f9d1549abb7a3cd001c8285702dc6d90132e3f31795583a: Status 404 returned error can't find the container with id cd6c46e02fa910044f9d1549abb7a3cd001c8285702dc6d90132e3f31795583a
Sep 14 02:52:26.702955 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.702925    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2"]
Sep 14 02:52:26.707070 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:52:26.707046    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34720b3c_3761_4c43_acda_d281e9b35cdc.slice/crio-746affd4f0e12d3233dc47016059e7729b10868cc06c1d377408b54d50fcdf55 WatchSource:0}: Error finding container 746affd4f0e12d3233dc47016059e7729b10868cc06c1d377408b54d50fcdf55: Status 404 returned error can't find the container with id 746affd4f0e12d3233dc47016059e7729b10868cc06c1d377408b54d50fcdf55
Sep 14 02:52:26.806109 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.806076    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2" event={"ID":"34720b3c-3761-4c43-acda-d281e9b35cdc","Type":"ContainerStarted","Data":"746affd4f0e12d3233dc47016059e7729b10868cc06c1d377408b54d50fcdf55"}
Sep 14 02:52:26.807698 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.807676    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p95kw" event={"ID":"fedea9f9-1caf-464e-94e1-31ca4f616882","Type":"ContainerStarted","Data":"0eef2b34e9ab04b05401b0fd8bf1be6dbe85a1be50a76ce810d8e29bc0916957"}
Sep 14 02:52:26.807801 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:26.807724    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p95kw" event={"ID":"fedea9f9-1caf-464e-94e1-31ca4f616882","Type":"ContainerStarted","Data":"cd6c46e02fa910044f9d1549abb7a3cd001c8285702dc6d90132e3f31795583a"}
Sep 14 02:52:27.813921 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:27.813874    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p95kw" event={"ID":"fedea9f9-1caf-464e-94e1-31ca4f616882","Type":"ContainerStarted","Data":"5cb59e0ef9c6ad816ad483ce1908b5294cfc1f7a6c025bbfdc0cc269a6f36f4a"}
Sep 14 02:52:30.826434 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:30.826402    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2" event={"ID":"34720b3c-3761-4c43-acda-d281e9b35cdc","Type":"ContainerStarted","Data":"c8b1cbf502653788c86cd0724b5543bd43b69b2ec21ae01b04f23c801cfce544"}
Sep 14 02:52:30.826809 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:30.826456    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2" event={"ID":"34720b3c-3761-4c43-acda-d281e9b35cdc","Type":"ContainerStarted","Data":"4093e2491f51e0c37256fc6f735528906f9bc5b24da66a312fffa4645090ed44"}
Sep 14 02:52:30.826809 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:30.826472    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2" event={"ID":"34720b3c-3761-4c43-acda-d281e9b35cdc","Type":"ContainerStarted","Data":"7edaaa2acac01117fe3e00281d98ffc74cc90197d7001d13d174e8276904d896"}
Sep 14 02:52:30.828744 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:30.828720    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p95kw" event={"ID":"fedea9f9-1caf-464e-94e1-31ca4f616882","Type":"ContainerStarted","Data":"f020bfb87217c2f179abe8cdde8ae82c4ceb21de56e2bd4bdcdd201342fc44cb"}
Sep 14 02:52:30.864491 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:30.864427    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-65dd976bc7-q7cf2" podStartSLOduration=0.919781305 podStartE2EDuration="4.864408797s" totalImagesPullingTime="3.944627492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:30.863199205 +0000 UTC m=+97.176223027" watchObservedRunningTime="2026-09-14 02:52:30.864408797 +0000 UTC m=+97.177432621"
Sep 14 02:52:30.887203 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:30.887102    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-p95kw" podStartSLOduration=1.005925666 podStartE2EDuration="4.887086422s" totalImagesPullingTime="3.881160756s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:30.886413606 +0000 UTC m=+97.199437425" watchObservedRunningTime="2026-09-14 02:52:30.887086422 +0000 UTC m=+97.200110241"
Sep 14 02:52:31.721567 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:31.721505    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fkpqf\" (UID: \"ff4eb2f8-e574-4e2c-a59f-0c715f466adb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:52:31.724027 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:31.723995    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff4eb2f8-e574-4e2c-a59f-0c715f466adb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-fkpqf\" (UID: \"ff4eb2f8-e574-4e2c-a59f-0c715f466adb\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:52:31.897626 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:31.897590    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"
Sep 14 02:52:32.171851 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:32.171829    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf"]
Sep 14 02:52:32.174388 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:52:32.174358    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff4eb2f8_e574_4e2c_a59f_0c715f466adb.slice/crio-eee6ec095f613656a7acf092c1dca625a2ecca1fb93ed429d1324d66e1911ec8 WatchSource:0}: Error finding container eee6ec095f613656a7acf092c1dca625a2ecca1fb93ed429d1324d66e1911ec8: Status 404 returned error can't find the container with id eee6ec095f613656a7acf092c1dca625a2ecca1fb93ed429d1324d66e1911ec8
Sep 14 02:52:32.837330 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:32.837293    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf" event={"ID":"ff4eb2f8-e574-4e2c-a59f-0c715f466adb","Type":"ContainerStarted","Data":"eee6ec095f613656a7acf092c1dca625a2ecca1fb93ed429d1324d66e1911ec8"}
Sep 14 02:52:32.930769 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:32.930674    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:52:32.933307 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:32.933285    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6a144e24-363d-48b1-a4a0-b27d07c109ba-metrics-tls\") pod \"dns-default-kxpfw\" (UID: \"6a144e24-363d-48b1-a4a0-b27d07c109ba\") " pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:52:32.963375 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:32.963343    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nzbvq\""
Sep 14 02:52:32.970923 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:32.970890    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:52:33.248594 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:33.248563    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kxpfw"]
Sep 14 02:52:33.251438 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:52:33.251411    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a144e24_363d_48b1_a4a0_b27d07c109ba.slice/crio-9cae1ca2788fa3bcd1bed14b13355b36072d4e67632d2aa14bc42195df554fb2 WatchSource:0}: Error finding container 9cae1ca2788fa3bcd1bed14b13355b36072d4e67632d2aa14bc42195df554fb2: Status 404 returned error can't find the container with id 9cae1ca2788fa3bcd1bed14b13355b36072d4e67632d2aa14bc42195df554fb2
Sep 14 02:52:33.850302 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:33.850270    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kxpfw" event={"ID":"6a144e24-363d-48b1-a4a0-b27d07c109ba","Type":"ContainerStarted","Data":"9cae1ca2788fa3bcd1bed14b13355b36072d4e67632d2aa14bc42195df554fb2"}
Sep 14 02:52:34.855825 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:34.855790    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf" event={"ID":"ff4eb2f8-e574-4e2c-a59f-0c715f466adb","Type":"ContainerStarted","Data":"23d589e9927e11ba2d4f15bfc266013955ec5daebb5efce187fe7d39388d3cef"}
Sep 14 02:52:34.875300 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:34.875238    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-fkpqf" podStartSLOduration=35.255718548 podStartE2EDuration="36.875219125s" totalImagesPullingTime="1.619500577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:34.873886162 +0000 UTC m=+101.186909981" watchObservedRunningTime="2026-09-14 02:52:34.875219125 +0000 UTC m=+101.188242943"
Sep 14 02:52:35.863541 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:35.863485    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kxpfw" event={"ID":"6a144e24-363d-48b1-a4a0-b27d07c109ba","Type":"ContainerStarted","Data":"ad9567def04a312b089a366f902e70472414f75284f50c9ae27de8f72a456ade"}
Sep 14 02:52:35.863955 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:35.863561    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kxpfw" event={"ID":"6a144e24-363d-48b1-a4a0-b27d07c109ba","Type":"ContainerStarted","Data":"51d04545ecccaa76731b0cbdb81f6dbb0fcd8a01e5adee776a04377ed7ff1b34"}
Sep 14 02:52:35.863955 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:35.863780    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:52:35.887305 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:35.887257    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kxpfw" podStartSLOduration=64.714350721 podStartE2EDuration="1m6.887243162s" totalImagesPullingTime="2.172892441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:51:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:35.887025924 +0000 UTC m=+102.200049743" watchObservedRunningTime="2026-09-14 02:52:35.887243162 +0000 UTC m=+102.200266980"
Sep 14 02:52:37.738690 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:37.738660    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5jzxs"
Sep 14 02:52:42.962260 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:42.962212    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cbwjn"]
Sep 14 02:52:42.965187 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:42.965165    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:42.973164 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:42.973144    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 14 02:52:42.973270 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:42.973227    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 14 02:52:42.973405 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:42.973387    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 14 02:52:42.973957 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:42.973943    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 14 02:52:42.973957 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:42.973952    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qnmp8\""
Sep 14 02:52:43.106982 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.106945    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-textfile\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.106982 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.106983    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-wtmp\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.107184 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.107003    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.107184 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.107031    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.107184 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.107068    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-root\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.107184 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.107096    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-tls\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.107184 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.107164    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl8tf\" (UniqueName: \"kubernetes.io/projected/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-kube-api-access-tl8tf\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.107357 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.107197    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-sys\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.107357 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.107211    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-metrics-client-ca\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.208247 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.208220    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-textfile\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.208388 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.208256    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-wtmp\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.208388 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.208275    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.208388 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.208301    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.208388 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.208326    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-root\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.208388 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.208347    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-tls\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.208388 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.208381    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl8tf\" (UniqueName: \"kubernetes.io/projected/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-kube-api-access-tl8tf\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.208699 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.208402    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-wtmp\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.208699 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.208444    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-sys\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.208699 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.208474    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-metrics-client-ca\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.208699 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:43.208564    2574 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 14 02:52:43.208699 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.208574    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-textfile\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.208699 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.208576    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-sys\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.208699 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.208635    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-root\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.208699 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:52:43.208687    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-tls podName:d0141b5c-8ece-4ae7-bf48-b756da7cf4c8 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:43.708663752 +0000 UTC m=+110.021687566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-tls") pod "node-exporter-cbwjn" (UID: "d0141b5c-8ece-4ae7-bf48-b756da7cf4c8") : secret "node-exporter-tls" not found
Sep 14 02:52:43.209034 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.208890    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.209034 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.208988    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-metrics-client-ca\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.210718 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.210702    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.217853 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.217798    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl8tf\" (UniqueName: \"kubernetes.io/projected/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-kube-api-access-tl8tf\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.712011 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.711979    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-tls\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.714325 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.714305    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d0141b5c-8ece-4ae7-bf48-b756da7cf4c8-node-exporter-tls\") pod \"node-exporter-cbwjn\" (UID: \"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8\") " pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.873918 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.873886    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cbwjn"
Sep 14 02:52:43.886812 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:52:43.886784    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0141b5c_8ece_4ae7_bf48_b756da7cf4c8.slice/crio-e0798daf6ff71ea59ed4a306635a24ffc7b1549539bf7caac03393571554423f WatchSource:0}: Error finding container e0798daf6ff71ea59ed4a306635a24ffc7b1549539bf7caac03393571554423f: Status 404 returned error can't find the container with id e0798daf6ff71ea59ed4a306635a24ffc7b1549539bf7caac03393571554423f
Sep 14 02:52:43.895445 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:43.895411    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cbwjn" event={"ID":"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8","Type":"ContainerStarted","Data":"e0798daf6ff71ea59ed4a306635a24ffc7b1549539bf7caac03393571554423f"}
Sep 14 02:52:45.868862 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:45.868833    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kxpfw"
Sep 14 02:52:45.905713 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:45.905685    2574 generic.go:356] "Generic (PLEG): container finished" podID="d0141b5c-8ece-4ae7-bf48-b756da7cf4c8" containerID="c950eaeacd244bd9a3e06015f8aa7eb8895d3bc2b863b5dbd80608d29039539f" exitCode=0
Sep 14 02:52:45.905860 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:45.905764    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cbwjn" event={"ID":"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8","Type":"ContainerDied","Data":"c950eaeacd244bd9a3e06015f8aa7eb8895d3bc2b863b5dbd80608d29039539f"}
Sep 14 02:52:46.913732 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:46.913696    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cbwjn" event={"ID":"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8","Type":"ContainerStarted","Data":"4857868340c913eb1f39697caa623295b8e79b118ddb7a591b4dab0d746d2b96"}
Sep 14 02:52:46.914104 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:46.913756    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cbwjn" event={"ID":"d0141b5c-8ece-4ae7-bf48-b756da7cf4c8","Type":"ContainerStarted","Data":"4b2a562b6878b60473670c8a2cc68dfdb90a6dc6b5ff52d6e3c830857efcf66f"}
Sep 14 02:52:46.936171 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:52:46.936127    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cbwjn" podStartSLOduration=3.863935347 podStartE2EDuration="4.936112393s" totalImagesPullingTime="1.004705834s" totalInitContainerRuntime="67.471212ms" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:46.934966753 +0000 UTC m=+113.247990572" watchObservedRunningTime="2026-09-14 02:52:46.936112393 +0000 UTC m=+113.249136211"
Sep 14 02:53:02.968773 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:02.968737    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs\") pod \"network-metrics-daemon-hkpjq\" (UID: \"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6\") " pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:53:02.971325 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:02.971294    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae964ae5-7b42-42c5-8b19-1bf113cfd6d6-metrics-certs\") pod \"network-metrics-daemon-hkpjq\" (UID: \"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6\") " pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:53:03.230996 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:03.230977    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mk9cr\""
Sep 14 02:53:03.239222 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:03.239199    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hkpjq"
Sep 14 02:53:03.567397 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:03.567323    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hkpjq"]
Sep 14 02:53:03.570961 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:53:03.570938    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae964ae5_7b42_42c5_8b19_1bf113cfd6d6.slice/crio-921d4040dcd26103c6fa054f7421d9a352f3c6ece7877d764877fe9fffb035f0 WatchSource:0}: Error finding container 921d4040dcd26103c6fa054f7421d9a352f3c6ece7877d764877fe9fffb035f0: Status 404 returned error can't find the container with id 921d4040dcd26103c6fa054f7421d9a352f3c6ece7877d764877fe9fffb035f0
Sep 14 02:53:03.974716 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:03.974617    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hkpjq" event={"ID":"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6","Type":"ContainerStarted","Data":"921d4040dcd26103c6fa054f7421d9a352f3c6ece7877d764877fe9fffb035f0"}
Sep 14 02:53:05.982952 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:05.982914    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hkpjq" event={"ID":"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6","Type":"ContainerStarted","Data":"e35b4a17109d3a326c96d1b4c0b3d8597f038433119c6578c40929193c221fbe"}
Sep 14 02:53:05.983338 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:05.982963    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hkpjq" event={"ID":"ae964ae5-7b42-42c5-8b19-1bf113cfd6d6","Type":"ContainerStarted","Data":"260f41b49c436d2c4423a8e31316295a476d96553a542d6ecda553a3233ec000"}
Sep 14 02:53:06.022538 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:06.022481    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hkpjq" podStartSLOduration=130.643317718 podStartE2EDuration="2m12.022467855s" totalImagesPullingTime="1.379150137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:50:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:06.021967889 +0000 UTC m=+132.334991706" watchObservedRunningTime="2026-09-14 02:53:06.022467855 +0000 UTC m=+132.335491673"
Sep 14 02:53:12.004368 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:12.004330    2574 generic.go:356] "Generic (PLEG): container finished" podID="c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b" containerID="50f85c431a0fbf760409ecb3c6503116fb4ce64be2855af10a1f9a2e8b929040" exitCode=0
Sep 14 02:53:12.004805 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:12.004415    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr" event={"ID":"c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b","Type":"ContainerDied","Data":"50f85c431a0fbf760409ecb3c6503116fb4ce64be2855af10a1f9a2e8b929040"}
Sep 14 02:53:12.004805 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:12.004765    2574 scope.go:117] "RemoveContainer" containerID="50f85c431a0fbf760409ecb3c6503116fb4ce64be2855af10a1f9a2e8b929040"
Sep 14 02:53:13.010138 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:13.010106    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-m9bnr" event={"ID":"c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b","Type":"ContainerStarted","Data":"e54a34539bedccd8bc15ac7d6acc9016c48207f989b4b52bc450b9307c7da665"}
Sep 14 02:53:38.103337 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:38.103303    2574 generic.go:356] "Generic (PLEG): container finished" podID="ae97ab3e-852d-4891-84bd-579e977e7aee" containerID="065bb31c674785384d0ba3de1cf5157d34c50d6f406b16895d24a64da7716158" exitCode=0
Sep 14 02:53:38.103748 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:38.103381    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-6gxb8" event={"ID":"ae97ab3e-852d-4891-84bd-579e977e7aee","Type":"ContainerDied","Data":"065bb31c674785384d0ba3de1cf5157d34c50d6f406b16895d24a64da7716158"}
Sep 14 02:53:38.103839 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:38.103823    2574 scope.go:117] "RemoveContainer" containerID="065bb31c674785384d0ba3de1cf5157d34c50d6f406b16895d24a64da7716158"
Sep 14 02:53:39.109312 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:53:39.109276    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-6gxb8" event={"ID":"ae97ab3e-852d-4891-84bd-579e977e7aee","Type":"ContainerStarted","Data":"61e76505794465402480ed940b0e20992d46602a478025f4bccce80d202af0bd"}
Sep 14 02:55:23.611617 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:23.611537    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxgjf"]
Sep 14 02:55:23.614602 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:23.614585    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxgjf"
Sep 14 02:55:23.621173 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:23.621152    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 14 02:55:23.621284 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:23.621152    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 14 02:55:23.621284 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:23.621219    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-lrkk2\""
Sep 14 02:55:23.655983 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:23.655956    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2p98\" (UniqueName: \"kubernetes.io/projected/19a1cf85-bf42-45f7-8316-244423381480-kube-api-access-k2p98\") pod \"cert-manager-operator-controller-manager-55c444bf86-jxgjf\" (UID: \"19a1cf85-bf42-45f7-8316-244423381480\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxgjf"
Sep 14 02:55:23.656093 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:23.655993    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19a1cf85-bf42-45f7-8316-244423381480-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jxgjf\" (UID: \"19a1cf85-bf42-45f7-8316-244423381480\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxgjf"
Sep 14 02:55:23.675501 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:23.675478    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxgjf"]
Sep 14 02:55:23.756581 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:23.756553    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2p98\" (UniqueName: \"kubernetes.io/projected/19a1cf85-bf42-45f7-8316-244423381480-kube-api-access-k2p98\") pod \"cert-manager-operator-controller-manager-55c444bf86-jxgjf\" (UID: \"19a1cf85-bf42-45f7-8316-244423381480\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxgjf"
Sep 14 02:55:23.756705 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:23.756603    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19a1cf85-bf42-45f7-8316-244423381480-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jxgjf\" (UID: \"19a1cf85-bf42-45f7-8316-244423381480\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxgjf"
Sep 14 02:55:23.756972 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:23.756952    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19a1cf85-bf42-45f7-8316-244423381480-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jxgjf\" (UID: \"19a1cf85-bf42-45f7-8316-244423381480\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxgjf"
Sep 14 02:55:23.772317 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:23.772288    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2p98\" (UniqueName: \"kubernetes.io/projected/19a1cf85-bf42-45f7-8316-244423381480-kube-api-access-k2p98\") pod \"cert-manager-operator-controller-manager-55c444bf86-jxgjf\" (UID: \"19a1cf85-bf42-45f7-8316-244423381480\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxgjf"
Sep 14 02:55:23.923555 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:23.923495    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxgjf"
Sep 14 02:55:24.224977 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:24.224952    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxgjf"]
Sep 14 02:55:24.229463 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:55:24.227709    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a1cf85_bf42_45f7_8316_244423381480.slice/crio-718cf76ea1730f9f9eb1926adaf60e4f650b0e775e7264a6c290da0facc6fd54 WatchSource:0}: Error finding container 718cf76ea1730f9f9eb1926adaf60e4f650b0e775e7264a6c290da0facc6fd54: Status 404 returned error can't find the container with id 718cf76ea1730f9f9eb1926adaf60e4f650b0e775e7264a6c290da0facc6fd54
Sep 14 02:55:24.480767 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:24.480732    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxgjf" event={"ID":"19a1cf85-bf42-45f7-8316-244423381480","Type":"ContainerStarted","Data":"718cf76ea1730f9f9eb1926adaf60e4f650b0e775e7264a6c290da0facc6fd54"}
Sep 14 02:55:27.493946 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:27.493912    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxgjf" event={"ID":"19a1cf85-bf42-45f7-8316-244423381480","Type":"ContainerStarted","Data":"64351abe12e0a35237d28636b38deb39d2f8434fd0c9e6e86f83dfdc5750b049"}
Sep 14 02:55:27.520359 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:27.520308    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxgjf" podStartSLOduration=1.377189054 podStartE2EDuration="4.520291996s" totalImagesPullingTime="3.143102942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:55:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:55:27.519958213 +0000 UTC m=+273.832982032" watchObservedRunningTime="2026-09-14 02:55:27.520291996 +0000 UTC m=+273.833315815"
Sep 14 02:55:35.009464 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:35.009430    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-m95xz"]
Sep 14 02:55:35.012629 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:35.012611    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-m95xz"
Sep 14 02:55:35.015485 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:35.015462    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-tftpk\""
Sep 14 02:55:35.015650 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:35.015462    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 14 02:55:35.016530 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:35.016497    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 14 02:55:35.025508 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:35.025487    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-m95xz"]
Sep 14 02:55:35.144829 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:35.144797    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w5g5\" (UniqueName: \"kubernetes.io/projected/f0940838-058e-4cd4-a1f5-9abf60e5afbd-kube-api-access-4w5g5\") pod \"cert-manager-webhook-6486ffd997-m95xz\" (UID: \"f0940838-058e-4cd4-a1f5-9abf60e5afbd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m95xz"
Sep 14 02:55:35.144990 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:35.144835    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0940838-058e-4cd4-a1f5-9abf60e5afbd-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-m95xz\" (UID: \"f0940838-058e-4cd4-a1f5-9abf60e5afbd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m95xz"
Sep 14 02:55:35.246007 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:35.245980    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w5g5\" (UniqueName: \"kubernetes.io/projected/f0940838-058e-4cd4-a1f5-9abf60e5afbd-kube-api-access-4w5g5\") pod \"cert-manager-webhook-6486ffd997-m95xz\" (UID: \"f0940838-058e-4cd4-a1f5-9abf60e5afbd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m95xz"
Sep 14 02:55:35.246144 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:35.246021    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0940838-058e-4cd4-a1f5-9abf60e5afbd-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-m95xz\" (UID: \"f0940838-058e-4cd4-a1f5-9abf60e5afbd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m95xz"
Sep 14 02:55:35.256862 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:35.256839    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0940838-058e-4cd4-a1f5-9abf60e5afbd-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-m95xz\" (UID: \"f0940838-058e-4cd4-a1f5-9abf60e5afbd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m95xz"
Sep 14 02:55:35.256976 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:35.256930    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w5g5\" (UniqueName: \"kubernetes.io/projected/f0940838-058e-4cd4-a1f5-9abf60e5afbd-kube-api-access-4w5g5\") pod \"cert-manager-webhook-6486ffd997-m95xz\" (UID: \"f0940838-058e-4cd4-a1f5-9abf60e5afbd\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m95xz"
Sep 14 02:55:35.321786 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:35.321701    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-m95xz"
Sep 14 02:55:35.591494 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:35.591470    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-m95xz"]
Sep 14 02:55:35.594062 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:55:35.594021    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0940838_058e_4cd4_a1f5_9abf60e5afbd.slice/crio-9d9b5e1a0e41e380fe5399b3acd8860832d13d21a6657b63c2f9479ae31fe2eb WatchSource:0}: Error finding container 9d9b5e1a0e41e380fe5399b3acd8860832d13d21a6657b63c2f9479ae31fe2eb: Status 404 returned error can't find the container with id 9d9b5e1a0e41e380fe5399b3acd8860832d13d21a6657b63c2f9479ae31fe2eb
Sep 14 02:55:36.529474 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:36.529438    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-m95xz" event={"ID":"f0940838-058e-4cd4-a1f5-9abf60e5afbd","Type":"ContainerStarted","Data":"9d9b5e1a0e41e380fe5399b3acd8860832d13d21a6657b63c2f9479ae31fe2eb"}
Sep 14 02:55:38.540002 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:38.539968    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-m95xz" event={"ID":"f0940838-058e-4cd4-a1f5-9abf60e5afbd","Type":"ContainerStarted","Data":"6b30188ecd0043fed60119104743d4c28f0e5988db09cdc9d5a9c85708c06349"}
Sep 14 02:55:38.540391 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:38.540207    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-m95xz"
Sep 14 02:55:38.560557 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:38.560496    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-m95xz" podStartSLOduration=2.187690901 podStartE2EDuration="4.560482852s" totalImagesPullingTime="2.372791951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:55:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:55:38.55899814 +0000 UTC m=+284.872021956" watchObservedRunningTime="2026-09-14 02:55:38.560482852 +0000 UTC m=+284.873506671"
Sep 14 02:55:44.546534 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:44.546487    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-m95xz"
Sep 14 02:55:54.094068 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:54.094039    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 02:55:54.096685 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:54.096660    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 02:55:54.102184 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:54.102165    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 02:55:54.104892 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:54.104875    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 02:55:54.106529 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:55:54.106493    2574 kubelet.go:1648] "Image garbage collection succeeded"
Sep 14 02:56:11.933479 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:11.933444    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"]
Sep 14 02:56:11.937098 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:11.937083    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:11.939884 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:11.939859    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 14 02:56:11.940018 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:11.939996    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 14 02:56:11.941217 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:11.941158    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2ts6x\""
Sep 14 02:56:11.941217 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:11.941206    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 14 02:56:11.941360 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:11.941218    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 14 02:56:11.941458 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:11.941445    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 14 02:56:11.948353 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:11.948332    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"]
Sep 14 02:56:12.017979 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.017954    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d-cert\") pod \"lws-controller-manager-5647bd797c-wcxbp\" (UID: \"2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d\") " pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:12.018060 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.017984    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t86bz\" (UniqueName: \"kubernetes.io/projected/2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d-kube-api-access-t86bz\") pod \"lws-controller-manager-5647bd797c-wcxbp\" (UID: \"2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d\") " pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:12.018060 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.018057    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d-metrics-cert\") pod \"lws-controller-manager-5647bd797c-wcxbp\" (UID: \"2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d\") " pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:12.018131 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.018075    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d-manager-config\") pod \"lws-controller-manager-5647bd797c-wcxbp\" (UID: \"2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d\") " pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:12.119303 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.119279    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d-metrics-cert\") pod \"lws-controller-manager-5647bd797c-wcxbp\" (UID: \"2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d\") " pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:12.119423 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.119308    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d-manager-config\") pod \"lws-controller-manager-5647bd797c-wcxbp\" (UID: \"2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d\") " pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:12.119423 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.119350    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d-cert\") pod \"lws-controller-manager-5647bd797c-wcxbp\" (UID: \"2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d\") " pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:12.119423 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.119374    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t86bz\" (UniqueName: \"kubernetes.io/projected/2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d-kube-api-access-t86bz\") pod \"lws-controller-manager-5647bd797c-wcxbp\" (UID: \"2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d\") " pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:12.120077 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.120000    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d-manager-config\") pod \"lws-controller-manager-5647bd797c-wcxbp\" (UID: \"2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d\") " pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:12.122013 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.121991    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d-cert\") pod \"lws-controller-manager-5647bd797c-wcxbp\" (UID: \"2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d\") " pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:12.122093 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.122054    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d-metrics-cert\") pod \"lws-controller-manager-5647bd797c-wcxbp\" (UID: \"2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d\") " pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:12.144257 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.144236    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t86bz\" (UniqueName: \"kubernetes.io/projected/2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d-kube-api-access-t86bz\") pod \"lws-controller-manager-5647bd797c-wcxbp\" (UID: \"2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d\") " pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:12.247529 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.247494    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:12.520621 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.520597    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"]
Sep 14 02:56:12.522544 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:56:12.522498    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d7b7c27_24d7_45ea_94b8_ff3e2bb37e5d.slice/crio-aa9a8eaf4abdf3236dce93a252569ea0876f8eab706c4183f68cc9c213c60e32 WatchSource:0}: Error finding container aa9a8eaf4abdf3236dce93a252569ea0876f8eab706c4183f68cc9c213c60e32: Status 404 returned error can't find the container with id aa9a8eaf4abdf3236dce93a252569ea0876f8eab706c4183f68cc9c213c60e32
Sep 14 02:56:12.524389 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.524373    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 02:56:12.676365 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:12.676334    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp" event={"ID":"2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d","Type":"ContainerStarted","Data":"aa9a8eaf4abdf3236dce93a252569ea0876f8eab706c4183f68cc9c213c60e32"}
Sep 14 02:56:15.692288 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:15.692253    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp" event={"ID":"2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d","Type":"ContainerStarted","Data":"c707e264d9490633b6f9635fed0e3af89d3f0180aa0888a0a86262c53fce7340"}
Sep 14 02:56:15.692717 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:15.692350    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:15.710503 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:15.710453    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp" podStartSLOduration=2.47730771 podStartE2EDuration="4.710439186s" totalImagesPullingTime="2.233131476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:56:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:56:15.709937396 +0000 UTC m=+322.022961214" watchObservedRunningTime="2026-09-14 02:56:15.710439186 +0000 UTC m=+322.023463005"
Sep 14 02:56:21.608645 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.608606    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"]
Sep 14 02:56:21.612018 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.611994    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.615302 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.615277    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 14 02:56:21.615445 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.615325    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 14 02:56:21.615537 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.615458    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q2bns\""
Sep 14 02:56:21.615595 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.615545    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 14 02:56:21.616492 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.616476    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 14 02:56:21.645200 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.645170    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"]
Sep 14 02:56:21.695686 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.695658    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ef0c507-14d9-4be4-9bc5-9d680635b0d9-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f647699c6-tjkbn\" (UID: \"0ef0c507-14d9-4be4-9bc5-9d680635b0d9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.695848 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.695700    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch666\" (UniqueName: \"kubernetes.io/projected/0ef0c507-14d9-4be4-9bc5-9d680635b0d9-kube-api-access-ch666\") pod \"opendatahub-operator-controller-manager-6f647699c6-tjkbn\" (UID: \"0ef0c507-14d9-4be4-9bc5-9d680635b0d9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.695848 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.695725    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0ef0c507-14d9-4be4-9bc5-9d680635b0d9-manifests\") pod \"opendatahub-operator-controller-manager-6f647699c6-tjkbn\" (UID: \"0ef0c507-14d9-4be4-9bc5-9d680635b0d9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.695848 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.695751    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ef0c507-14d9-4be4-9bc5-9d680635b0d9-tmp\") pod \"opendatahub-operator-controller-manager-6f647699c6-tjkbn\" (UID: \"0ef0c507-14d9-4be4-9bc5-9d680635b0d9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.695848 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.695776    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0ef0c507-14d9-4be4-9bc5-9d680635b0d9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f647699c6-tjkbn\" (UID: \"0ef0c507-14d9-4be4-9bc5-9d680635b0d9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.796794 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.796755    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ef0c507-14d9-4be4-9bc5-9d680635b0d9-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f647699c6-tjkbn\" (UID: \"0ef0c507-14d9-4be4-9bc5-9d680635b0d9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.796986 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.796811    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch666\" (UniqueName: \"kubernetes.io/projected/0ef0c507-14d9-4be4-9bc5-9d680635b0d9-kube-api-access-ch666\") pod \"opendatahub-operator-controller-manager-6f647699c6-tjkbn\" (UID: \"0ef0c507-14d9-4be4-9bc5-9d680635b0d9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.796986 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.796843    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0ef0c507-14d9-4be4-9bc5-9d680635b0d9-manifests\") pod \"opendatahub-operator-controller-manager-6f647699c6-tjkbn\" (UID: \"0ef0c507-14d9-4be4-9bc5-9d680635b0d9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.796986 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.796874    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ef0c507-14d9-4be4-9bc5-9d680635b0d9-tmp\") pod \"opendatahub-operator-controller-manager-6f647699c6-tjkbn\" (UID: \"0ef0c507-14d9-4be4-9bc5-9d680635b0d9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.796986 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.796906    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0ef0c507-14d9-4be4-9bc5-9d680635b0d9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f647699c6-tjkbn\" (UID: \"0ef0c507-14d9-4be4-9bc5-9d680635b0d9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.797322 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.797292    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0ef0c507-14d9-4be4-9bc5-9d680635b0d9-manifests\") pod \"opendatahub-operator-controller-manager-6f647699c6-tjkbn\" (UID: \"0ef0c507-14d9-4be4-9bc5-9d680635b0d9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.799251 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.799226    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ef0c507-14d9-4be4-9bc5-9d680635b0d9-tmp\") pod \"opendatahub-operator-controller-manager-6f647699c6-tjkbn\" (UID: \"0ef0c507-14d9-4be4-9bc5-9d680635b0d9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.799371 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.799351    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ef0c507-14d9-4be4-9bc5-9d680635b0d9-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f647699c6-tjkbn\" (UID: \"0ef0c507-14d9-4be4-9bc5-9d680635b0d9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.799571 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.799552    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0ef0c507-14d9-4be4-9bc5-9d680635b0d9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f647699c6-tjkbn\" (UID: \"0ef0c507-14d9-4be4-9bc5-9d680635b0d9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.825143 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.825106    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch666\" (UniqueName: \"kubernetes.io/projected/0ef0c507-14d9-4be4-9bc5-9d680635b0d9-kube-api-access-ch666\") pod \"opendatahub-operator-controller-manager-6f647699c6-tjkbn\" (UID: \"0ef0c507-14d9-4be4-9bc5-9d680635b0d9\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:21.921821 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:21.921731    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:22.214436 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:56:22.214402    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ef0c507_14d9_4be4_9bc5_9d680635b0d9.slice/crio-2329acfc11e05d37db89d91f8180532da43891ae79ba7d759d8979b84f2ba5fe WatchSource:0}: Error finding container 2329acfc11e05d37db89d91f8180532da43891ae79ba7d759d8979b84f2ba5fe: Status 404 returned error can't find the container with id 2329acfc11e05d37db89d91f8180532da43891ae79ba7d759d8979b84f2ba5fe
Sep 14 02:56:22.215489 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:22.215468    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"]
Sep 14 02:56:22.722078 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:22.722039    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn" event={"ID":"0ef0c507-14d9-4be4-9bc5-9d680635b0d9","Type":"ContainerStarted","Data":"2329acfc11e05d37db89d91f8180532da43891ae79ba7d759d8979b84f2ba5fe"}
Sep 14 02:56:25.737595 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:25.737552    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn" event={"ID":"0ef0c507-14d9-4be4-9bc5-9d680635b0d9","Type":"ContainerStarted","Data":"f27f28a01c3be174beadb53d9b4909e55a2ffe6fb52fc154f73452e02c584a5d"}
Sep 14 02:56:26.699467 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:26.699367    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5647bd797c-wcxbp"
Sep 14 02:56:26.744382 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:26.744348    2574 generic.go:356] "Generic (PLEG): container finished" podID="0ef0c507-14d9-4be4-9bc5-9d680635b0d9" containerID="f27f28a01c3be174beadb53d9b4909e55a2ffe6fb52fc154f73452e02c584a5d" exitCode=0
Sep 14 02:56:26.744820 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:26.744398    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn" event={"ID":"0ef0c507-14d9-4be4-9bc5-9d680635b0d9","Type":"ContainerDied","Data":"f27f28a01c3be174beadb53d9b4909e55a2ffe6fb52fc154f73452e02c584a5d"}
Sep 14 02:56:27.754839 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:27.754799    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn" event={"ID":"0ef0c507-14d9-4be4-9bc5-9d680635b0d9","Type":"ContainerStarted","Data":"db927f389a887eaa08fc2bbd184c11c8e0526fa77d6366641c5d1c11d0e75a20"}
Sep 14 02:56:27.755324 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:27.754896    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:27.780755 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:27.780703    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn" podStartSLOduration=2.4860658239999998 podStartE2EDuration="6.78068545s" totalImagesPullingTime="3.212719699s" totalInitContainerRuntime="1.081899927s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:56:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:56:27.779351592 +0000 UTC m=+334.092375411" watchObservedRunningTime="2026-09-14 02:56:27.78068545 +0000 UTC m=+334.093709270"
Sep 14 02:56:38.760776 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:38.760739    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-tjkbn"
Sep 14 02:56:48.985843 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:48.985762    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"]
Sep 14 02:56:48.988959 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:48.988937    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:48.992494 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:48.992470    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 14 02:56:48.992705 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:48.992588    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 14 02:56:48.992705 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:48.992475    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-6jkns\""
Sep 14 02:56:48.992966 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:48.992949    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 02:56:49.005452 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.005426    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"]
Sep 14 02:56:49.115722 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.115683    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72ad94c5-039e-4ea7-a293-5aa10ed606b8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.115722 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.115723    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72ad94c5-039e-4ea7-a293-5aa10ed606b8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.115952 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.115741    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffk7d\" (UniqueName: \"kubernetes.io/projected/72ad94c5-039e-4ea7-a293-5aa10ed606b8-kube-api-access-ffk7d\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.115952 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.115844    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72ad94c5-039e-4ea7-a293-5aa10ed606b8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.115952 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.115883    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72ad94c5-039e-4ea7-a293-5aa10ed606b8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.115952 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.115908    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72ad94c5-039e-4ea7-a293-5aa10ed606b8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.116113 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.115995    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72ad94c5-039e-4ea7-a293-5aa10ed606b8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.116113 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.116033    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72ad94c5-039e-4ea7-a293-5aa10ed606b8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.116113 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.116067    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72ad94c5-039e-4ea7-a293-5aa10ed606b8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.216883 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.216835    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72ad94c5-039e-4ea7-a293-5aa10ed606b8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.216883 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.216880    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72ad94c5-039e-4ea7-a293-5aa10ed606b8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.217137 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.216898    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffk7d\" (UniqueName: \"kubernetes.io/projected/72ad94c5-039e-4ea7-a293-5aa10ed606b8-kube-api-access-ffk7d\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.217137 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.216949    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72ad94c5-039e-4ea7-a293-5aa10ed606b8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.217137 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.216975    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72ad94c5-039e-4ea7-a293-5aa10ed606b8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.217137 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.217002    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72ad94c5-039e-4ea7-a293-5aa10ed606b8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.217137 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.217032    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72ad94c5-039e-4ea7-a293-5aa10ed606b8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.217137 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.217056    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72ad94c5-039e-4ea7-a293-5aa10ed606b8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.217137 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.217084    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72ad94c5-039e-4ea7-a293-5aa10ed606b8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.217554 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.217297    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72ad94c5-039e-4ea7-a293-5aa10ed606b8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.217554 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.217397    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72ad94c5-039e-4ea7-a293-5aa10ed606b8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.217554 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.217455    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72ad94c5-039e-4ea7-a293-5aa10ed606b8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.217554 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.217505    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72ad94c5-039e-4ea7-a293-5aa10ed606b8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.217782 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.217752    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72ad94c5-039e-4ea7-a293-5aa10ed606b8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.219575 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.219556    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72ad94c5-039e-4ea7-a293-5aa10ed606b8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.219910 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.219889    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72ad94c5-039e-4ea7-a293-5aa10ed606b8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.230616 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.230593    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72ad94c5-039e-4ea7-a293-5aa10ed606b8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.230710 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.230615    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffk7d\" (UniqueName: \"kubernetes.io/projected/72ad94c5-039e-4ea7-a293-5aa10ed606b8-kube-api-access-ffk7d\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87\" (UID: \"72ad94c5-039e-4ea7-a293-5aa10ed606b8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.299573 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.299468    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:49.577987 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.577962    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"]
Sep 14 02:56:49.580237 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:56:49.580215    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ad94c5_039e_4ea7_a293_5aa10ed606b8.slice/crio-5ca395d352b3f19b5e8279eef1bd189bed5333d532fdbf36a05346a95e157dc5 WatchSource:0}: Error finding container 5ca395d352b3f19b5e8279eef1bd189bed5333d532fdbf36a05346a95e157dc5: Status 404 returned error can't find the container with id 5ca395d352b3f19b5e8279eef1bd189bed5333d532fdbf36a05346a95e157dc5
Sep 14 02:56:49.846584 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:49.846483    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87" event={"ID":"72ad94c5-039e-4ea7-a293-5aa10ed606b8","Type":"ContainerStarted","Data":"5ca395d352b3f19b5e8279eef1bd189bed5333d532fdbf36a05346a95e157dc5"}
Sep 14 02:56:52.283442 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:52.283390    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 02:56:52.283782 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:52.283493    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 02:56:52.283782 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:52.283581    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 02:56:52.861555 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:52.861495    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87" event={"ID":"72ad94c5-039e-4ea7-a293-5aa10ed606b8","Type":"ContainerStarted","Data":"8d3e26daf7adea1931e6cef65ba06d5f8c7ee4f525d94ce3d189861e8f38e95c"}
Sep 14 02:56:53.300457 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:53.300415    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:53.305718 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:53.305689    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:53.329071 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:53.329014    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87" podStartSLOduration=2.628044873 podStartE2EDuration="5.32899893s" totalImagesPullingTime="2.700954057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:56:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:56:52.887223857 +0000 UTC m=+359.200247675" watchObservedRunningTime="2026-09-14 02:56:53.32899893 +0000 UTC m=+359.642022750"
Sep 14 02:56:53.866364 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:53.866323    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:56:53.867499 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:56:53.867478    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87"
Sep 14 02:57:02.593244 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:02.593211    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl"]
Sep 14 02:57:02.597888 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:02.597868    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl"
Sep 14 02:57:02.601154 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:02.601132    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 14 02:57:02.601154 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:02.601149    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-m8vf9\""
Sep 14 02:57:02.638420 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:02.638382    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl"]
Sep 14 02:57:02.736647 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:02.736603    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swbpg\" (UniqueName: \"kubernetes.io/projected/d37c2e20-7fd9-4719-9b81-1f35e2f9e62c-kube-api-access-swbpg\") pod \"kserve-controller-manager-d6f4b65b8-mnsrl\" (UID: \"d37c2e20-7fd9-4719-9b81-1f35e2f9e62c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl"
Sep 14 02:57:02.736811 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:02.736690    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d37c2e20-7fd9-4719-9b81-1f35e2f9e62c-cert\") pod \"kserve-controller-manager-d6f4b65b8-mnsrl\" (UID: \"d37c2e20-7fd9-4719-9b81-1f35e2f9e62c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl"
Sep 14 02:57:02.837487 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:02.837457    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d37c2e20-7fd9-4719-9b81-1f35e2f9e62c-cert\") pod \"kserve-controller-manager-d6f4b65b8-mnsrl\" (UID: \"d37c2e20-7fd9-4719-9b81-1f35e2f9e62c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl"
Sep 14 02:57:02.837646 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:02.837509    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swbpg\" (UniqueName: \"kubernetes.io/projected/d37c2e20-7fd9-4719-9b81-1f35e2f9e62c-kube-api-access-swbpg\") pod \"kserve-controller-manager-d6f4b65b8-mnsrl\" (UID: \"d37c2e20-7fd9-4719-9b81-1f35e2f9e62c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl"
Sep 14 02:57:02.837646 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:57:02.837637    2574 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 14 02:57:02.837740 ip-10-0-133-188 kubenswrapper[2574]: E0914 02:57:02.837701    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d37c2e20-7fd9-4719-9b81-1f35e2f9e62c-cert podName:d37c2e20-7fd9-4719-9b81-1f35e2f9e62c nodeName:}" failed. No retries permitted until 2026-09-14 02:57:03.337684951 +0000 UTC m=+369.650708753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d37c2e20-7fd9-4719-9b81-1f35e2f9e62c-cert") pod "kserve-controller-manager-d6f4b65b8-mnsrl" (UID: "d37c2e20-7fd9-4719-9b81-1f35e2f9e62c") : secret "kserve-webhook-server-cert" not found
Sep 14 02:57:02.872096 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:02.872032    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swbpg\" (UniqueName: \"kubernetes.io/projected/d37c2e20-7fd9-4719-9b81-1f35e2f9e62c-kube-api-access-swbpg\") pod \"kserve-controller-manager-d6f4b65b8-mnsrl\" (UID: \"d37c2e20-7fd9-4719-9b81-1f35e2f9e62c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl"
Sep 14 02:57:03.341623 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:03.341578    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d37c2e20-7fd9-4719-9b81-1f35e2f9e62c-cert\") pod \"kserve-controller-manager-d6f4b65b8-mnsrl\" (UID: \"d37c2e20-7fd9-4719-9b81-1f35e2f9e62c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl"
Sep 14 02:57:03.344263 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:03.344241    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d37c2e20-7fd9-4719-9b81-1f35e2f9e62c-cert\") pod \"kserve-controller-manager-d6f4b65b8-mnsrl\" (UID: \"d37c2e20-7fd9-4719-9b81-1f35e2f9e62c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl"
Sep 14 02:57:03.508374 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:03.508333    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl"
Sep 14 02:57:03.812551 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:03.812501    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl"]
Sep 14 02:57:03.815209 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:57:03.815183    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd37c2e20_7fd9_4719_9b81_1f35e2f9e62c.slice/crio-c263d05aaeaebe2161a4dcb68bfc622304fb2d8d0ecc6ae6595c94c2545d22d9 WatchSource:0}: Error finding container c263d05aaeaebe2161a4dcb68bfc622304fb2d8d0ecc6ae6595c94c2545d22d9: Status 404 returned error can't find the container with id c263d05aaeaebe2161a4dcb68bfc622304fb2d8d0ecc6ae6595c94c2545d22d9
Sep 14 02:57:03.911283 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:03.911249    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl" event={"ID":"d37c2e20-7fd9-4719-9b81-1f35e2f9e62c","Type":"ContainerStarted","Data":"c263d05aaeaebe2161a4dcb68bfc622304fb2d8d0ecc6ae6595c94c2545d22d9"}
Sep 14 02:57:06.927442 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:06.927401    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl" event={"ID":"d37c2e20-7fd9-4719-9b81-1f35e2f9e62c","Type":"ContainerStarted","Data":"d474cb035f0df5b80aa8b6234da6c493ac6105f56e456b60e5637cefcc9fd15f"}
Sep 14 02:57:06.928009 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:06.927499    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl"
Sep 14 02:57:06.946711 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:06.946662    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl" podStartSLOduration=2.395482568 podStartE2EDuration="4.946646954s" totalImagesPullingTime="2.551164386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:57:06.945384181 +0000 UTC m=+373.258408000" watchObservedRunningTime="2026-09-14 02:57:06.946646954 +0000 UTC m=+373.259670773"
Sep 14 02:57:37.936991 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:57:37.936955    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnsrl"
Sep 14 02:58:13.769453 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:13.769419    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-tj9tz"]
Sep 14 02:58:13.772773 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:13.772752    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-tj9tz"
Sep 14 02:58:13.775301 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:13.775277    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 14 02:58:13.775406 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:13.775343    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 14 02:58:13.775670 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:13.775653    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5nlrn\""
Sep 14 02:58:13.794447 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:13.794425    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-tj9tz"]
Sep 14 02:58:13.902873 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:13.902846    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8lt6\" (UniqueName: \"kubernetes.io/projected/5e0482aa-e17e-4624-8aa2-c9d6f2bf4e70-kube-api-access-d8lt6\") pod \"authorino-operator-76b478bd78-tj9tz\" (UID: \"5e0482aa-e17e-4624-8aa2-c9d6f2bf4e70\") " pod="kuadrant-system/authorino-operator-76b478bd78-tj9tz"
Sep 14 02:58:14.003217 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:14.003192    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8lt6\" (UniqueName: \"kubernetes.io/projected/5e0482aa-e17e-4624-8aa2-c9d6f2bf4e70-kube-api-access-d8lt6\") pod \"authorino-operator-76b478bd78-tj9tz\" (UID: \"5e0482aa-e17e-4624-8aa2-c9d6f2bf4e70\") " pod="kuadrant-system/authorino-operator-76b478bd78-tj9tz"
Sep 14 02:58:14.017644 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:14.017618    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8lt6\" (UniqueName: \"kubernetes.io/projected/5e0482aa-e17e-4624-8aa2-c9d6f2bf4e70-kube-api-access-d8lt6\") pod \"authorino-operator-76b478bd78-tj9tz\" (UID: \"5e0482aa-e17e-4624-8aa2-c9d6f2bf4e70\") " pod="kuadrant-system/authorino-operator-76b478bd78-tj9tz"
Sep 14 02:58:14.082567 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:14.082485    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-tj9tz"
Sep 14 02:58:14.377036 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:14.376928    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-tj9tz"]
Sep 14 02:58:14.379948 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:58:14.379922    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0482aa_e17e_4624_8aa2_c9d6f2bf4e70.slice/crio-a779b80abccab2c648b9ae774a802fdc52a61f1cd266bed3b6437fd1b8d33a19 WatchSource:0}: Error finding container a779b80abccab2c648b9ae774a802fdc52a61f1cd266bed3b6437fd1b8d33a19: Status 404 returned error can't find the container with id a779b80abccab2c648b9ae774a802fdc52a61f1cd266bed3b6437fd1b8d33a19
Sep 14 02:58:15.220485 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:15.220455    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-tj9tz" event={"ID":"5e0482aa-e17e-4624-8aa2-c9d6f2bf4e70","Type":"ContainerStarted","Data":"a779b80abccab2c648b9ae774a802fdc52a61f1cd266bed3b6437fd1b8d33a19"}
Sep 14 02:58:18.236460 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:18.236431    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-tj9tz" event={"ID":"5e0482aa-e17e-4624-8aa2-c9d6f2bf4e70","Type":"ContainerStarted","Data":"3d4fcb38363f21bf79110a90f9f98d41e7da2e00dbf74ad21472b90acea1d712"}
Sep 14 02:58:18.236854 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:18.236580    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-tj9tz"
Sep 14 02:58:27.575601 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.575539    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-tj9tz" podStartSLOduration=11.149233668 podStartE2EDuration="14.575503172s" totalImagesPullingTime="3.426269504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:58:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:58:18.271909608 +0000 UTC m=+444.584933427" watchObservedRunningTime="2026-09-14 02:58:27.575503172 +0000 UTC m=+453.888526994"
Sep 14 02:58:27.576474 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.576452    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc"]
Sep 14 02:58:27.579715 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.579696    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc"
Sep 14 02:58:27.583697 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.583680    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 14 02:58:27.583841 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.583823    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 14 02:58:27.583915 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.583831    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wmm8d\""
Sep 14 02:58:27.598468 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.598445    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc"]
Sep 14 02:58:27.613503 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.613478    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j8xn\" (UniqueName: \"kubernetes.io/projected/bd6b9db2-2a69-4d03-b442-ba6308955f3f-kube-api-access-9j8xn\") pod \"kuadrant-console-plugin-fbdd4f445-k7gcc\" (UID: \"bd6b9db2-2a69-4d03-b442-ba6308955f3f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc"
Sep 14 02:58:27.613646 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.613534    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd6b9db2-2a69-4d03-b442-ba6308955f3f-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-k7gcc\" (UID: \"bd6b9db2-2a69-4d03-b442-ba6308955f3f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc"
Sep 14 02:58:27.613646 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.613624    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bd6b9db2-2a69-4d03-b442-ba6308955f3f-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-k7gcc\" (UID: \"bd6b9db2-2a69-4d03-b442-ba6308955f3f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc"
Sep 14 02:58:27.714223 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.714182    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bd6b9db2-2a69-4d03-b442-ba6308955f3f-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-k7gcc\" (UID: \"bd6b9db2-2a69-4d03-b442-ba6308955f3f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc"
Sep 14 02:58:27.714408 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.714246    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j8xn\" (UniqueName: \"kubernetes.io/projected/bd6b9db2-2a69-4d03-b442-ba6308955f3f-kube-api-access-9j8xn\") pod \"kuadrant-console-plugin-fbdd4f445-k7gcc\" (UID: \"bd6b9db2-2a69-4d03-b442-ba6308955f3f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc"
Sep 14 02:58:27.714408 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.714268    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd6b9db2-2a69-4d03-b442-ba6308955f3f-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-k7gcc\" (UID: \"bd6b9db2-2a69-4d03-b442-ba6308955f3f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc"
Sep 14 02:58:27.714983 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.714961    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bd6b9db2-2a69-4d03-b442-ba6308955f3f-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-k7gcc\" (UID: \"bd6b9db2-2a69-4d03-b442-ba6308955f3f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc"
Sep 14 02:58:27.716952 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.716927    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd6b9db2-2a69-4d03-b442-ba6308955f3f-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-k7gcc\" (UID: \"bd6b9db2-2a69-4d03-b442-ba6308955f3f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc"
Sep 14 02:58:27.735068 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.735038    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j8xn\" (UniqueName: \"kubernetes.io/projected/bd6b9db2-2a69-4d03-b442-ba6308955f3f-kube-api-access-9j8xn\") pod \"kuadrant-console-plugin-fbdd4f445-k7gcc\" (UID: \"bd6b9db2-2a69-4d03-b442-ba6308955f3f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc"
Sep 14 02:58:27.889088 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:27.888992    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc"
Sep 14 02:58:28.164161 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:28.164138    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc"]
Sep 14 02:58:28.165089 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:58:28.165065    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd6b9db2_2a69_4d03_b442_ba6308955f3f.slice/crio-24f1e801c3934353ef93b2a2241c795d63cda0ef216449c4eb59839ab4356a1d WatchSource:0}: Error finding container 24f1e801c3934353ef93b2a2241c795d63cda0ef216449c4eb59839ab4356a1d: Status 404 returned error can't find the container with id 24f1e801c3934353ef93b2a2241c795d63cda0ef216449c4eb59839ab4356a1d
Sep 14 02:58:28.277568 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:28.277536    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc" event={"ID":"bd6b9db2-2a69-4d03-b442-ba6308955f3f","Type":"ContainerStarted","Data":"24f1e801c3934353ef93b2a2241c795d63cda0ef216449c4eb59839ab4356a1d"}
Sep 14 02:58:29.243048 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:29.243015    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-tj9tz"
Sep 14 02:58:33.311097 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:33.311064    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc" event={"ID":"bd6b9db2-2a69-4d03-b442-ba6308955f3f","Type":"ContainerStarted","Data":"c53b12c67db1738c0d9b6248ded385d7bff99e833c9e0613bb19dffd4412d95f"}
Sep 14 02:58:33.360419 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:58:33.360358    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k7gcc" podStartSLOduration=1.947675406 podStartE2EDuration="6.36034216s" totalImagesPullingTime="4.412666754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:58:33.358418595 +0000 UTC m=+459.671442414" watchObservedRunningTime="2026-09-14 02:58:33.36034216 +0000 UTC m=+459.673365979"
Sep 14 02:59:12.325415 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.325384    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"]
Sep 14 02:59:12.348615 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.348570    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.348775 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.348657    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"]
Sep 14 02:59:12.352119 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.352097    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-7gjjw\""
Sep 14 02:59:12.453253 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.453230    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.453394 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.453269    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.453394 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.453290    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.453394 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.453307    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rmxc\" (UniqueName: \"kubernetes.io/projected/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-kube-api-access-4rmxc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.453394 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.453374    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.453595 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.453429    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.453595 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.453478    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.453595 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.453501    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.453595 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.453586    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.554069 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.554046    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.554196 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.554077    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.554196 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.554099    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.554196 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.554116    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.554196 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.554134    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.554396 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.554258    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.554396 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.554325    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.554396 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.554362    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.554396 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.554391    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rmxc\" (UniqueName: \"kubernetes.io/projected/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-kube-api-access-4rmxc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.554646 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.554626    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.554703 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.554691    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.554805 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.554783    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.554918 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.554897    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.555062 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.555037    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.556736 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.556716    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.556997 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.556978    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.568697 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.568670    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.568785 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.568728    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rmxc\" (UniqueName: \"kubernetes.io/projected/ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795-kube-api-access-4rmxc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hdvvq\" (UID: \"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.659317 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.659262    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:12.944043 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.944014    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"]
Sep 14 02:59:12.945143 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:59:12.945118    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea0da5e4_6cc3_4bf1_99bb_a9b46ad9f795.slice/crio-4074727a6a7932dd688e21a1710ab3c696c5f67eff8a6c9f2f5c28952db23c93 WatchSource:0}: Error finding container 4074727a6a7932dd688e21a1710ab3c696c5f67eff8a6c9f2f5c28952db23c93: Status 404 returned error can't find the container with id 4074727a6a7932dd688e21a1710ab3c696c5f67eff8a6c9f2f5c28952db23c93
Sep 14 02:59:12.947271 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.947244    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 02:59:12.947331 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.947303    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 02:59:12.947368 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:12.947333    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 02:59:13.482563 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:13.482527    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq" event={"ID":"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795","Type":"ContainerStarted","Data":"0345913192e0e6ce41f6e8198c2da1955b200d758139b7e333f1ba673f7b685d"}
Sep 14 02:59:13.482947 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:13.482582    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq" event={"ID":"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795","Type":"ContainerStarted","Data":"4074727a6a7932dd688e21a1710ab3c696c5f67eff8a6c9f2f5c28952db23c93"}
Sep 14 02:59:13.507419 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:13.507370    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq" podStartSLOduration=1.5073561199999999 podStartE2EDuration="1.50735612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:59:13.504301156 +0000 UTC m=+499.817324987" watchObservedRunningTime="2026-09-14 02:59:13.50735612 +0000 UTC m=+499.820379940"
Sep 14 02:59:13.659917 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:13.659889    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:13.664609 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:13.664586    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:14.487118 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:14.487086    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:14.488171 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:14.488148    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 02:59:14.900123 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:14.900044    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 02:59:14.903407 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:14.903390    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"
Sep 14 02:59:14.906074 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:14.906056    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 14 02:59:14.913710 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:14.913683    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 02:59:14.943153 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:14.943131    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 02:59:14.975928 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:14.975901    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cfbd2a72-d324-456f-be02-832d3a442bdb-config-file\") pod \"limitador-limitador-84994f7ffc-p2cbh\" (UID: \"cfbd2a72-d324-456f-be02-832d3a442bdb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"
Sep 14 02:59:14.976041 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:14.975953    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqj69\" (UniqueName: \"kubernetes.io/projected/cfbd2a72-d324-456f-be02-832d3a442bdb-kube-api-access-zqj69\") pod \"limitador-limitador-84994f7ffc-p2cbh\" (UID: \"cfbd2a72-d324-456f-be02-832d3a442bdb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"
Sep 14 02:59:15.076689 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:15.076665    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cfbd2a72-d324-456f-be02-832d3a442bdb-config-file\") pod \"limitador-limitador-84994f7ffc-p2cbh\" (UID: \"cfbd2a72-d324-456f-be02-832d3a442bdb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"
Sep 14 02:59:15.076783 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:15.076721    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqj69\" (UniqueName: \"kubernetes.io/projected/cfbd2a72-d324-456f-be02-832d3a442bdb-kube-api-access-zqj69\") pod \"limitador-limitador-84994f7ffc-p2cbh\" (UID: \"cfbd2a72-d324-456f-be02-832d3a442bdb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"
Sep 14 02:59:15.077246 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:15.077227    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cfbd2a72-d324-456f-be02-832d3a442bdb-config-file\") pod \"limitador-limitador-84994f7ffc-p2cbh\" (UID: \"cfbd2a72-d324-456f-be02-832d3a442bdb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"
Sep 14 02:59:15.086477 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:15.086455    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqj69\" (UniqueName: \"kubernetes.io/projected/cfbd2a72-d324-456f-be02-832d3a442bdb-kube-api-access-zqj69\") pod \"limitador-limitador-84994f7ffc-p2cbh\" (UID: \"cfbd2a72-d324-456f-be02-832d3a442bdb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"
Sep 14 02:59:15.214455 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:15.214390    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"
Sep 14 02:59:15.489347 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:15.489316    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 02:59:15.491647 ip-10-0-133-188 kubenswrapper[2574]: W0914 02:59:15.491622    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbd2a72_d324_456f_be02_832d3a442bdb.slice/crio-e656f721ed4844992478e74cc27d8aff767b9828fc3fb306a28061cbef30c441 WatchSource:0}: Error finding container e656f721ed4844992478e74cc27d8aff767b9828fc3fb306a28061cbef30c441: Status 404 returned error can't find the container with id e656f721ed4844992478e74cc27d8aff767b9828fc3fb306a28061cbef30c441
Sep 14 02:59:16.498684 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:16.498648    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-p2cbh" event={"ID":"cfbd2a72-d324-456f-be02-832d3a442bdb","Type":"ContainerStarted","Data":"e656f721ed4844992478e74cc27d8aff767b9828fc3fb306a28061cbef30c441"}
Sep 14 02:59:17.504958 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:17.504924    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-p2cbh" event={"ID":"cfbd2a72-d324-456f-be02-832d3a442bdb","Type":"ContainerStarted","Data":"6c041539c90bca8c90fc93619efca6bc9dbe1c9107c655f7ba001ba4c8a7427d"}
Sep 14 02:59:17.505445 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:17.505049    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"
Sep 14 02:59:17.525157 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:17.525106    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-p2cbh" podStartSLOduration=2.259999791 podStartE2EDuration="3.525092639s" totalImagesPullingTime="1.265092848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:59:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:59:17.524449101 +0000 UTC m=+503.837472920" watchObservedRunningTime="2026-09-14 02:59:17.525092639 +0000 UTC m=+503.838116457"
Sep 14 02:59:28.510053 ip-10-0-133-188 kubenswrapper[2574]: I0914 02:59:28.510020    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"
Sep 14 03:00:29.094795 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:29.094755    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:00:37.868180 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:37.868150    2574 generic.go:356] "Generic (PLEG): container finished" podID="ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795" containerID="0345913192e0e6ce41f6e8198c2da1955b200d758139b7e333f1ba673f7b685d" exitCode=0
Sep 14 03:00:37.868530 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:37.868192    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq" event={"ID":"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795","Type":"ContainerDied","Data":"0345913192e0e6ce41f6e8198c2da1955b200d758139b7e333f1ba673f7b685d"}
Sep 14 03:00:37.868530 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:37.868505    2574 scope.go:117] "RemoveContainer" containerID="0345913192e0e6ce41f6e8198c2da1955b200d758139b7e333f1ba673f7b685d"
Sep 14 03:00:37.869460 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:37.869433    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:00:37.869541 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:37.869502    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:00:37.869579 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:37.869564    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:00:38.877831 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:38.877783    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq" event={"ID":"ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795","Type":"ContainerStarted","Data":"207f31b2bf8c7ddbb24ee71d5fd125dbcd2bd9c6ab753bd652af458e0696b40b"}
Sep 14 03:00:39.659680 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:39.659639    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 03:00:39.659877 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:39.659839    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hdvvq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 14 03:00:39.659925 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:39.659895    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq" podUID="ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 14 03:00:40.660602 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:40.660567    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hdvvq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 14 03:00:40.660990 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:40.660642    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq" podUID="ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 14 03:00:41.660382 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:41.660348    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hdvvq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 14 03:00:41.660593 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:41.660416    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq" podUID="ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 14 03:00:42.660586 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:42.660554    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hdvvq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 14 03:00:42.660953 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:42.660625    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq" podUID="ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 14 03:00:43.659976 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:43.659940    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hdvvq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 14 03:00:43.660152 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:43.659999    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq" podUID="ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 14 03:00:44.488032 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:44.487996    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 03:00:44.663968 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:44.663937    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 03:00:44.912671 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:44.912582    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq"
Sep 14 03:00:54.124460 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:54.124425    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 03:00:54.124894 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:54.124541    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 03:00:54.131122 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:54.131091    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 03:00:54.131340 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:00:54.131323    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 03:01:08.643531 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:08.643480    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:01:12.656747 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:12.656709    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:01:36.134895 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:36.134861    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:01:46.251144 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:46.251107    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:01:56.580897 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.580856    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"]
Sep 14 03:01:56.584744 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.584721    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.588865 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.588844    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 14 03:01:56.589012 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.588867    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 14 03:01:56.589012 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.588846    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hcz4h\""
Sep 14 03:01:56.589012 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.588844    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 14 03:01:56.597450 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.597423    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"]
Sep 14 03:01:56.778092 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.778051    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45f32a8f-1036-4441-897b-9ca88164b64e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.778092 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.778095    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45f32a8f-1036-4441-897b-9ca88164b64e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.778361 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.778111    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45f32a8f-1036-4441-897b-9ca88164b64e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.778361 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.778131    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5vkc\" (UniqueName: \"kubernetes.io/projected/45f32a8f-1036-4441-897b-9ca88164b64e-kube-api-access-x5vkc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.778361 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.778209    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45f32a8f-1036-4441-897b-9ca88164b64e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.778361 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.778304    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45f32a8f-1036-4441-897b-9ca88164b64e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.879429 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.879334    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45f32a8f-1036-4441-897b-9ca88164b64e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.879429 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.879389    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45f32a8f-1036-4441-897b-9ca88164b64e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.879429 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.879431    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45f32a8f-1036-4441-897b-9ca88164b64e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.879763 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.879451    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45f32a8f-1036-4441-897b-9ca88164b64e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.879763 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.879469    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45f32a8f-1036-4441-897b-9ca88164b64e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.879763 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.879576    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5vkc\" (UniqueName: \"kubernetes.io/projected/45f32a8f-1036-4441-897b-9ca88164b64e-kube-api-access-x5vkc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.879913 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.879856    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45f32a8f-1036-4441-897b-9ca88164b64e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.879913 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.879893    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45f32a8f-1036-4441-897b-9ca88164b64e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.880013 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.879921    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45f32a8f-1036-4441-897b-9ca88164b64e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.881948 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.881929    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45f32a8f-1036-4441-897b-9ca88164b64e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.882217 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.882200    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45f32a8f-1036-4441-897b-9ca88164b64e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.888409 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.888383    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5vkc\" (UniqueName: \"kubernetes.io/projected/45f32a8f-1036-4441-897b-9ca88164b64e-kube-api-access-x5vkc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s\" (UID: \"45f32a8f-1036-4441-897b-9ca88164b64e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:56.895263 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:56.895238    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:01:57.169071 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:57.169043    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"]
Sep 14 03:01:57.170942 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:01:57.170915    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45f32a8f_1036_4441_897b_9ca88164b64e.slice/crio-ffb6adb761ba649428de0b82456987f04e262183b0bd9977eeece8477dd3a5b4 WatchSource:0}: Error finding container ffb6adb761ba649428de0b82456987f04e262183b0bd9977eeece8477dd3a5b4: Status 404 returned error can't find the container with id ffb6adb761ba649428de0b82456987f04e262183b0bd9977eeece8477dd3a5b4
Sep 14 03:01:57.172320 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:57.172299    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 03:01:57.254772 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:57.254731    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s" event={"ID":"45f32a8f-1036-4441-897b-9ca88164b64e","Type":"ContainerStarted","Data":"ffb6adb761ba649428de0b82456987f04e262183b0bd9977eeece8477dd3a5b4"}
Sep 14 03:01:58.137711 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:58.137685    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:01:58.260663 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:58.260621    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s" event={"ID":"45f32a8f-1036-4441-897b-9ca88164b64e","Type":"ContainerStarted","Data":"81648f7a03b5bcba8e6d2a6adec0ce255db579ea1a06038f064d5a6cfdbe16ce"}
Sep 14 03:01:58.284869 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:01:58.284814    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s" podStartSLOduration=1.326851595 podStartE2EDuration="2.284795511s" totalImagesPullingTime="957.943916ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:01:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:01:58.282990705 +0000 UTC m=+664.596014524" watchObservedRunningTime="2026-09-14 03:01:58.284795511 +0000 UTC m=+664.597819347"
Sep 14 03:02:06.896320 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:06.896285    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:02:06.896904 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:06.896345    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:02:06.908911 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:06.908882    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:02:07.313672 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:07.313642    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqtj5s"
Sep 14 03:02:14.933773 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:14.933724    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:02:30.397257 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.397221    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"]
Sep 14 03:02:30.400632 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.400607    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.403638 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.403618    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 14 03:02:30.411495 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.411476    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"]
Sep 14 03:02:30.444706 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.444681    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98771704-d31c-43a8-b1b7-f64c4857efa0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.444801 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.444710    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98771704-d31c-43a8-b1b7-f64c4857efa0-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.444801 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.444725    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98771704-d31c-43a8-b1b7-f64c4857efa0-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.444801 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.444747    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98771704-d31c-43a8-b1b7-f64c4857efa0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.444950 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.444851    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr6jx\" (UniqueName: \"kubernetes.io/projected/98771704-d31c-43a8-b1b7-f64c4857efa0-kube-api-access-tr6jx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.444950 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.444932    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98771704-d31c-43a8-b1b7-f64c4857efa0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.545274 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.545246    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98771704-d31c-43a8-b1b7-f64c4857efa0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.545383 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.545310    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98771704-d31c-43a8-b1b7-f64c4857efa0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.545383 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.545340    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98771704-d31c-43a8-b1b7-f64c4857efa0-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.545383 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.545365    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98771704-d31c-43a8-b1b7-f64c4857efa0-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.545573 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.545398    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98771704-d31c-43a8-b1b7-f64c4857efa0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.545573 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.545434    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr6jx\" (UniqueName: \"kubernetes.io/projected/98771704-d31c-43a8-b1b7-f64c4857efa0-kube-api-access-tr6jx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.545856 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.545829    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98771704-d31c-43a8-b1b7-f64c4857efa0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.545955 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.545900    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98771704-d31c-43a8-b1b7-f64c4857efa0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.545955 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.545931    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98771704-d31c-43a8-b1b7-f64c4857efa0-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.547924 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.547904    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98771704-d31c-43a8-b1b7-f64c4857efa0-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.548090 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.548073    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98771704-d31c-43a8-b1b7-f64c4857efa0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.554834 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.554803    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr6jx\" (UniqueName: \"kubernetes.io/projected/98771704-d31c-43a8-b1b7-f64c4857efa0-kube-api-access-tr6jx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-g77cw\" (UID: \"98771704-d31c-43a8-b1b7-f64c4857efa0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.709894 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.709837    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:30.985152 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:30.985126    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"]
Sep 14 03:02:30.987159 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:02:30.987135    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98771704_d31c_43a8_b1b7_f64c4857efa0.slice/crio-f58aab846284b53524f31896f7c914a6ad89366c2aaf825e01edf4cfb79e78d3 WatchSource:0}: Error finding container f58aab846284b53524f31896f7c914a6ad89366c2aaf825e01edf4cfb79e78d3: Status 404 returned error can't find the container with id f58aab846284b53524f31896f7c914a6ad89366c2aaf825e01edf4cfb79e78d3
Sep 14 03:02:31.411291 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:31.411214    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw" event={"ID":"98771704-d31c-43a8-b1b7-f64c4857efa0","Type":"ContainerStarted","Data":"96d71e06613f520ee1766be62e72e7edaac8245190a5c9270b6363760d72023a"}
Sep 14 03:02:31.411291 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:31.411255    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw" event={"ID":"98771704-d31c-43a8-b1b7-f64c4857efa0","Type":"ContainerStarted","Data":"f58aab846284b53524f31896f7c914a6ad89366c2aaf825e01edf4cfb79e78d3"}
Sep 14 03:02:31.436606 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:31.436562    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw" podStartSLOduration=1.265635378 podStartE2EDuration="1.436546043s" totalImagesPullingTime="170.910665ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:02:31.435401478 +0000 UTC m=+697.748425308" watchObservedRunningTime="2026-09-14 03:02:31.436546043 +0000 UTC m=+697.749569863"
Sep 14 03:02:40.710152 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:40.710112    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:40.710646 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:40.710171    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:40.722924 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:40.722900    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:02:41.466048 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:02:41.466016    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-g77cw"
Sep 14 03:03:33.956689 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:03:33.956649    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:04:38.040381 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.040297    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"]
Sep 14 03:04:38.045012 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.044982    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.048483 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.048458    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e467de-openshift-default-dockercfg-blnr5\""
Sep 14 03:04:38.058487 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.058465    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"]
Sep 14 03:04:38.098969 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.098938    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-istio-envoy\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.099101 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.098987    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-workload-certs\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.099101 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.099075    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7edbe9e7-db64-421a-9330-98c14982d896-istio-podinfo\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.099185 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.099117    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7edbe9e7-db64-421a-9330-98c14982d896-istio-token\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.099185 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.099138    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-workload-socket\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.099185 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.099161    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gjjk\" (UniqueName: \"kubernetes.io/projected/7edbe9e7-db64-421a-9330-98c14982d896-kube-api-access-7gjjk\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.099282 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.099193    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-istio-data\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.099282 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.099209    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7edbe9e7-db64-421a-9330-98c14982d896-istiod-ca-cert\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.099282 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.099241    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-credential-socket\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.200268 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.200234    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7edbe9e7-db64-421a-9330-98c14982d896-istio-token\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.200411 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.200284    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-workload-socket\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.200411 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.200392    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gjjk\" (UniqueName: \"kubernetes.io/projected/7edbe9e7-db64-421a-9330-98c14982d896-kube-api-access-7gjjk\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.200493 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.200445    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-istio-data\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.200702 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.200665    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7edbe9e7-db64-421a-9330-98c14982d896-istiod-ca-cert\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.200932 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.200905    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-credential-socket\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.201067 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.200713    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-workload-socket\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.201067 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.200752    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-istio-data\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.201067 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.201004    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-istio-envoy\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.201067 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.201042    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-workload-certs\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.201293 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.201086    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7edbe9e7-db64-421a-9330-98c14982d896-istio-podinfo\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.201293 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.201226    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-credential-socket\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.201293 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.201228    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7edbe9e7-db64-421a-9330-98c14982d896-istiod-ca-cert\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.201453 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.201394    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-workload-certs\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.203358 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.203337    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-istio-envoy\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.203655 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.203639    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7edbe9e7-db64-421a-9330-98c14982d896-istio-podinfo\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.210050 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.210026    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7edbe9e7-db64-421a-9330-98c14982d896-istio-token\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.210271 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.210247    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gjjk\" (UniqueName: \"kubernetes.io/projected/7edbe9e7-db64-421a-9330-98c14982d896-kube-api-access-7gjjk\") pod \"e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") " pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.357067 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.356990    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:38.660807 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.660778    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"]
Sep 14 03:04:38.662305 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:04:38.662279    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7edbe9e7_db64_421a_9330_98c14982d896.slice/crio-d1be016b6391ff3a0153cfba6df011984d1c4800c446b44fcf094836562a07fc WatchSource:0}: Error finding container d1be016b6391ff3a0153cfba6df011984d1c4800c446b44fcf094836562a07fc: Status 404 returned error can't find the container with id d1be016b6391ff3a0153cfba6df011984d1c4800c446b44fcf094836562a07fc
Sep 14 03:04:38.664470 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.664440    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:04:38.664609 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.664501    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:04:38.664609 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.664547    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:04:38.995753 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.995716    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4" event={"ID":"7edbe9e7-db64-421a-9330-98c14982d896","Type":"ContainerStarted","Data":"11654c6a755fd96a610bc8f62740d4275d0560427786092eadaa457d0a53e64b"}
Sep 14 03:04:38.995909 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:38.995766    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4" event={"ID":"7edbe9e7-db64-421a-9330-98c14982d896","Type":"ContainerStarted","Data":"d1be016b6391ff3a0153cfba6df011984d1c4800c446b44fcf094836562a07fc"}
Sep 14 03:04:39.024667 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:39.024619    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4" podStartSLOduration=1.024603211 podStartE2EDuration="1.024603211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:04:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:04:39.023364582 +0000 UTC m=+825.336388400" watchObservedRunningTime="2026-09-14 03:04:39.024603211 +0000 UTC m=+825.337627069"
Sep 14 03:04:39.357939 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:39.357867    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:39.362174 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:39.362151    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:40.000942 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:40.000911    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:40.001843 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:40.001827    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:04:44.479688 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:44.479649    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9005-9b375d"]
Sep 14 03:04:44.483214 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:44.483197    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9005-9b375d"
Sep 14 03:04:44.486241 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:44.486215    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4ff8v\""
Sep 14 03:04:44.492926 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:44.492901    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9005-9b375d"]
Sep 14 03:04:44.554990 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:44.554960    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvjmj\" (UniqueName: \"kubernetes.io/projected/f2a9a0cd-8903-4b77-9515-dc5480eedaf3-kube-api-access-zvjmj\") pod \"test-curl-9005-9b375d\" (UID: \"f2a9a0cd-8903-4b77-9515-dc5480eedaf3\") " pod="opendatahub/test-curl-9005-9b375d"
Sep 14 03:04:44.655403 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:44.655372    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvjmj\" (UniqueName: \"kubernetes.io/projected/f2a9a0cd-8903-4b77-9515-dc5480eedaf3-kube-api-access-zvjmj\") pod \"test-curl-9005-9b375d\" (UID: \"f2a9a0cd-8903-4b77-9515-dc5480eedaf3\") " pod="opendatahub/test-curl-9005-9b375d"
Sep 14 03:04:44.666255 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:44.666225    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvjmj\" (UniqueName: \"kubernetes.io/projected/f2a9a0cd-8903-4b77-9515-dc5480eedaf3-kube-api-access-zvjmj\") pod \"test-curl-9005-9b375d\" (UID: \"f2a9a0cd-8903-4b77-9515-dc5480eedaf3\") " pod="opendatahub/test-curl-9005-9b375d"
Sep 14 03:04:44.793092 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:44.793013    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9005-9b375d"
Sep 14 03:04:45.087844 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:45.087821    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9005-9b375d"]
Sep 14 03:04:45.089806 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:04:45.089776    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2a9a0cd_8903_4b77_9515_dc5480eedaf3.slice/crio-c74acd4cf7d80f1c1069c9a4d9fb2bb80f54cd4fb74fd3f311789fca867742dd WatchSource:0}: Error finding container c74acd4cf7d80f1c1069c9a4d9fb2bb80f54cd4fb74fd3f311789fca867742dd: Status 404 returned error can't find the container with id c74acd4cf7d80f1c1069c9a4d9fb2bb80f54cd4fb74fd3f311789fca867742dd
Sep 14 03:04:46.034780 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:46.034745    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9005-9b375d" event={"ID":"f2a9a0cd-8903-4b77-9515-dc5480eedaf3","Type":"ContainerStarted","Data":"c74acd4cf7d80f1c1069c9a4d9fb2bb80f54cd4fb74fd3f311789fca867742dd"}
Sep 14 03:04:47.499451 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.499421    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"]
Sep 14 03:04:47.502849 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.502830    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.508571 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.508551    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c4b97c19-openshift-default-dockercfg-s4b6t\""
Sep 14 03:04:47.528215 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.528189    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"]
Sep 14 03:04:47.582938 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.582890    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21bae7db-ef7f-4897-8fbf-9f6923e01173-istiod-ca-cert\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.583136 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.583022    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-credential-socket\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.583136 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.583055    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-workload-socket\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.583136 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.583081    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4286j\" (UniqueName: \"kubernetes.io/projected/21bae7db-ef7f-4897-8fbf-9f6923e01173-kube-api-access-4286j\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.583136 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.583108    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-workload-certs\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.583136 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.583131    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-envoy\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.583773 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.583200    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-token\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.583773 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.583222    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-podinfo\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.583773 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.583257    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-data\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.684106 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.684059    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-credential-socket\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.684106 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.684108    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-workload-socket\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.684344 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.684156    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4286j\" (UniqueName: \"kubernetes.io/projected/21bae7db-ef7f-4897-8fbf-9f6923e01173-kube-api-access-4286j\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.684344 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.684197    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-workload-certs\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.684344 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.684213    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-envoy\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.684344 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.684251    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-token\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.684344 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.684274    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-podinfo\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.684344 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.684290    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-data\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.684712 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.684361    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21bae7db-ef7f-4897-8fbf-9f6923e01173-istiod-ca-cert\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.684712 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.684489    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-credential-socket\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.684712 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.684569    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-workload-socket\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.684815 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.684721    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-workload-certs\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.684887 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.684869    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-data\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.685040 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.685019    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21bae7db-ef7f-4897-8fbf-9f6923e01173-istiod-ca-cert\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.686753 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.686723    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-envoy\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.687130 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.687112    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-podinfo\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.702074 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.702045    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4286j\" (UniqueName: \"kubernetes.io/projected/21bae7db-ef7f-4897-8fbf-9f6923e01173-kube-api-access-4286j\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.703238 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.703218    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-token\") pod \"e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") " pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:47.765220 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.765138    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:04:47.813360 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:47.813329    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:48.047335 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:48.047237    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9005-9b375d" event={"ID":"f2a9a0cd-8903-4b77-9515-dc5480eedaf3","Type":"ContainerStarted","Data":"541e3192728ffe6874bdeaa6e24c965f6ba941a2b90a1de9e332d5a2bf5cdbf7"}
Sep 14 03:04:48.062924 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:48.062871    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-9005-9b375d" podStartSLOduration=1.937772173 podStartE2EDuration="4.062856219s" totalImagesPullingTime="2.125084046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:04:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:04:48.062339561 +0000 UTC m=+834.375363380" watchObservedRunningTime="2026-09-14 03:04:48.062856219 +0000 UTC m=+834.375880038"
Sep 14 03:04:48.093101 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:48.093080    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"]
Sep 14 03:04:48.095076 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:04:48.095048    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21bae7db_ef7f_4897_8fbf_9f6923e01173.slice/crio-fb369417f1b153123f0fb1a768c950a958327428ddfc8c7cfa890cd0b5fa7b35 WatchSource:0}: Error finding container fb369417f1b153123f0fb1a768c950a958327428ddfc8c7cfa890cd0b5fa7b35: Status 404 returned error can't find the container with id fb369417f1b153123f0fb1a768c950a958327428ddfc8c7cfa890cd0b5fa7b35
Sep 14 03:04:48.097155 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:48.097127    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:04:48.097236 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:48.097193    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:04:48.097236 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:48.097224    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:04:48.698975 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:48.698938    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9005-9b375d"]
Sep 14 03:04:48.707733 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:48.707661    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9005-9b375d"]
Sep 14 03:04:49.053973 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:49.053935    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz" event={"ID":"21bae7db-ef7f-4897-8fbf-9f6923e01173","Type":"ContainerStarted","Data":"8f7f81d460e9c45202c3a0ae7645fab7f56d4bd47953a1349df4ac08f73ab5ce"}
Sep 14 03:04:49.054164 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:49.053986    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz" event={"ID":"21bae7db-ef7f-4897-8fbf-9f6923e01173","Type":"ContainerStarted","Data":"fb369417f1b153123f0fb1a768c950a958327428ddfc8c7cfa890cd0b5fa7b35"}
Sep 14 03:04:49.054164 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:04:49.054139    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-9005-9b375d" container="test-curl-9005-9b375d"
Sep 14 03:04:49.081234 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:49.081170    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz" podStartSLOduration=2.08115037 podStartE2EDuration="2.08115037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:04:49.079704451 +0000 UTC m=+835.392728272" watchObservedRunningTime="2026-09-14 03:04:49.08115037 +0000 UTC m=+835.394174189"
Sep 14 03:04:49.814556 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:49.814495    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:49.819545 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:49.819506    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:50.058674 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:50.058629    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-9005-9b375d" podUID="f2a9a0cd-8903-4b77-9515-dc5480eedaf3" containerName="test-curl-9005-9b375d" containerID="cri-o://541e3192728ffe6874bdeaa6e24c965f6ba941a2b90a1de9e332d5a2bf5cdbf7" gracePeriod=2
Sep 14 03:04:50.058895 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:50.058855    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:50.059767 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:50.059750    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:04:50.061114 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:04:50.061095    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9005-9b375d\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" podUID="f2a9a0cd-8903-4b77-9515-dc5480eedaf3" pod="opendatahub/test-curl-9005-9b375d"
Sep 14 03:04:50.063045 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:04:50.063018    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9005-9b375d\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" podUID="f2a9a0cd-8903-4b77-9515-dc5480eedaf3" pod="opendatahub/test-curl-9005-9b375d"
Sep 14 03:04:52.287012 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:52.286819    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9005-9b375d"
Sep 14 03:04:52.426068 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:52.425981    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvjmj\" (UniqueName: \"kubernetes.io/projected/f2a9a0cd-8903-4b77-9515-dc5480eedaf3-kube-api-access-zvjmj\") pod \"f2a9a0cd-8903-4b77-9515-dc5480eedaf3\" (UID: \"f2a9a0cd-8903-4b77-9515-dc5480eedaf3\") "
Sep 14 03:04:52.428351 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:52.428328    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2a9a0cd-8903-4b77-9515-dc5480eedaf3-kube-api-access-zvjmj" (OuterVolumeSpecName: "kube-api-access-zvjmj") pod "f2a9a0cd-8903-4b77-9515-dc5480eedaf3" (UID: "f2a9a0cd-8903-4b77-9515-dc5480eedaf3"). InnerVolumeSpecName "kube-api-access-zvjmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:04:52.526945 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:52.526905    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvjmj\" (UniqueName: \"kubernetes.io/projected/f2a9a0cd-8903-4b77-9515-dc5480eedaf3-kube-api-access-zvjmj\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:04:53.096876 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:53.096837    2574 generic.go:356] "Generic (PLEG): container finished" podID="f2a9a0cd-8903-4b77-9515-dc5480eedaf3" containerID="541e3192728ffe6874bdeaa6e24c965f6ba941a2b90a1de9e332d5a2bf5cdbf7" exitCode=137
Sep 14 03:04:53.097069 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:53.096888    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9005-9b375d"
Sep 14 03:04:53.097069 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:53.096964    2574 scope.go:117] "RemoveContainer" containerID="541e3192728ffe6874bdeaa6e24c965f6ba941a2b90a1de9e332d5a2bf5cdbf7"
Sep 14 03:04:53.106725 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:53.106705    2574 scope.go:117] "RemoveContainer" containerID="541e3192728ffe6874bdeaa6e24c965f6ba941a2b90a1de9e332d5a2bf5cdbf7"
Sep 14 03:04:53.107054 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:04:53.107033    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"541e3192728ffe6874bdeaa6e24c965f6ba941a2b90a1de9e332d5a2bf5cdbf7\": container with ID starting with 541e3192728ffe6874bdeaa6e24c965f6ba941a2b90a1de9e332d5a2bf5cdbf7 not found: ID does not exist" containerID="541e3192728ffe6874bdeaa6e24c965f6ba941a2b90a1de9e332d5a2bf5cdbf7"
Sep 14 03:04:53.107103 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:53.107062    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"541e3192728ffe6874bdeaa6e24c965f6ba941a2b90a1de9e332d5a2bf5cdbf7"} err="failed to get container status \"541e3192728ffe6874bdeaa6e24c965f6ba941a2b90a1de9e332d5a2bf5cdbf7\": rpc error: code = NotFound desc = could not find container \"541e3192728ffe6874bdeaa6e24c965f6ba941a2b90a1de9e332d5a2bf5cdbf7\": container with ID starting with 541e3192728ffe6874bdeaa6e24c965f6ba941a2b90a1de9e332d5a2bf5cdbf7 not found: ID does not exist"
Sep 14 03:04:53.109075 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:04:53.109052    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9005-9b375d\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" podUID="f2a9a0cd-8903-4b77-9515-dc5480eedaf3" pod="opendatahub/test-curl-9005-9b375d"
Sep 14 03:04:54.215633 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:54.215604    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2a9a0cd-8903-4b77-9515-dc5480eedaf3" path="/var/lib/kubelet/pods/f2a9a0cd-8903-4b77-9515-dc5480eedaf3/volumes"
Sep 14 03:04:54.216743 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:04:54.216710    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9005-9b375d\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" podUID="f2a9a0cd-8903-4b77-9515-dc5480eedaf3" pod="opendatahub/test-curl-9005-9b375d"
Sep 14 03:04:56.441575 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.441543    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"]
Sep 14 03:04:56.441928 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.441871    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2a9a0cd-8903-4b77-9515-dc5480eedaf3" containerName="test-curl-9005-9b375d"
Sep 14 03:04:56.441928 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.441882    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2a9a0cd-8903-4b77-9515-dc5480eedaf3" containerName="test-curl-9005-9b375d"
Sep 14 03:04:56.441994 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.441961    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2a9a0cd-8903-4b77-9515-dc5480eedaf3" containerName="test-curl-9005-9b375d"
Sep 14 03:04:56.446203 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.446184    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.449986 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.449962    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-bc1117-openshift-default-dockercfg-2cnxp\""
Sep 14 03:04:56.463708 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.463683    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"]
Sep 14 03:04:56.565434 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.565403    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-credential-socket\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.565434 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.565435    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-envoy\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.565775 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.565467    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-podinfo\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.565775 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.565549    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-data\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.565775 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.565619    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgcb8\" (UniqueName: \"kubernetes.io/projected/426574c6-8729-47fd-bc40-2c612dfcd7ac-kube-api-access-xgcb8\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.565775 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.565710    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/426574c6-8729-47fd-bc40-2c612dfcd7ac-istiod-ca-cert\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.565775 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.565750    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-workload-certs\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.566041 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.565806    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-workload-socket\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.566041 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.565840    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-token\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.673538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.669974    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-workload-socket\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.673538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.670031    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-token\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.673538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.670097    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-credential-socket\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.673538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.670123    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-envoy\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.673538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.670155    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-podinfo\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.673538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.670181    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-data\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.673538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.670220    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgcb8\" (UniqueName: \"kubernetes.io/projected/426574c6-8729-47fd-bc40-2c612dfcd7ac-kube-api-access-xgcb8\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.673538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.670266    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/426574c6-8729-47fd-bc40-2c612dfcd7ac-istiod-ca-cert\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.673538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.670303    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-workload-certs\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.673538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.670806    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-workload-certs\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.673538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.671804    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-data\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.673538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.672290    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-workload-socket\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.673538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.672545    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-credential-socket\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.673538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.673179    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/426574c6-8729-47fd-bc40-2c612dfcd7ac-istiod-ca-cert\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.678541 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.675612    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-envoy\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.688538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.683817    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgcb8\" (UniqueName: \"kubernetes.io/projected/426574c6-8729-47fd-bc40-2c612dfcd7ac-kube-api-access-xgcb8\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.695111 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.695042    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-token\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.696132 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.696077    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-podinfo\") pod \"e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") " pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:56.756338 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:56.756304    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:57.063812 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:57.063779    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"]
Sep 14 03:04:57.065652 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:04:57.065627    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod426574c6_8729_47fd_bc40_2c612dfcd7ac.slice/crio-e568591be4bb61b55c3a24f49d27225a588c3809fcfe203b84c1114f540317af WatchSource:0}: Error finding container e568591be4bb61b55c3a24f49d27225a588c3809fcfe203b84c1114f540317af: Status 404 returned error can't find the container with id e568591be4bb61b55c3a24f49d27225a588c3809fcfe203b84c1114f540317af
Sep 14 03:04:57.068426 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:57.068393    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:04:57.068506 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:57.068467    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:04:57.068591 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:57.068529    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:04:57.118265 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:57.118230    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f" event={"ID":"426574c6-8729-47fd-bc40-2c612dfcd7ac","Type":"ContainerStarted","Data":"e568591be4bb61b55c3a24f49d27225a588c3809fcfe203b84c1114f540317af"}
Sep 14 03:04:58.124569 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:58.124528    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f" event={"ID":"426574c6-8729-47fd-bc40-2c612dfcd7ac","Type":"ContainerStarted","Data":"fb9826212932a644196f4902ac373b14af3e92745d63d5b907e8f7ffd80892fc"}
Sep 14 03:04:58.151576 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:58.151495    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f" podStartSLOduration=2.151477941 podStartE2EDuration="2.151477941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:04:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:04:58.151302851 +0000 UTC m=+844.464326684" watchObservedRunningTime="2026-09-14 03:04:58.151477941 +0000 UTC m=+844.464501759"
Sep 14 03:04:58.757179 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:58.757141    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:58.763052 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:58.763019    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:59.130078 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:59.130008    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:04:59.131048 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:04:59.131025    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:05:00.136506 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:00.136474    2574 generic.go:356] "Generic (PLEG): container finished" podID="7edbe9e7-db64-421a-9330-98c14982d896" containerID="11654c6a755fd96a610bc8f62740d4275d0560427786092eadaa457d0a53e64b" exitCode=0
Sep 14 03:05:00.136910 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:00.136551    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4" event={"ID":"7edbe9e7-db64-421a-9330-98c14982d896","Type":"ContainerDied","Data":"11654c6a755fd96a610bc8f62740d4275d0560427786092eadaa457d0a53e64b"}
Sep 14 03:05:00.137080 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:00.137063    2574 scope.go:117] "RemoveContainer" containerID="11654c6a755fd96a610bc8f62740d4275d0560427786092eadaa457d0a53e64b"
Sep 14 03:05:00.138183 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:00.138157    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:05:00.138252 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:00.138225    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:05:00.138291 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:00.138262    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:05:00.653238 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:00.653202    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:05:01.143782 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:01.143748    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4" event={"ID":"7edbe9e7-db64-421a-9330-98c14982d896","Type":"ContainerStarted","Data":"1c147cc95563049040f23626a8c374e11ed9c8c73a91d5196b924b97f9a2fb57"}
Sep 14 03:05:01.357989 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:01.357953    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:05:01.358297 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:01.358275    2574 patch_prober.go:28] interesting pod/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 14 03:05:01.358384 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:01.358329    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4" podUID="7edbe9e7-db64-421a-9330-98c14982d896" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 14 03:05:02.358370 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:02.358325    2574 patch_prober.go:28] interesting pod/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 14 03:05:02.358892 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:02.358403    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4" podUID="7edbe9e7-db64-421a-9330-98c14982d896" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 14 03:05:03.358382 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:03.358343    2574 patch_prober.go:28] interesting pod/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 14 03:05:03.358825 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:03.358412    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4" podUID="7edbe9e7-db64-421a-9330-98c14982d896" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 14 03:05:04.358353 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:04.358317    2574 patch_prober.go:28] interesting pod/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 14 03:05:04.358566 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:04.358385    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4" podUID="7edbe9e7-db64-421a-9330-98c14982d896" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 14 03:05:05.358347 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:05.358312    2574 patch_prober.go:28] interesting pod/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 14 03:05:05.358536 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:05.358373    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4" podUID="7edbe9e7-db64-421a-9330-98c14982d896" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 14 03:05:06.357828 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:06.357785    2574 patch_prober.go:28] interesting pod/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 14 03:05:06.358248 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:06.357852    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4" podUID="7edbe9e7-db64-421a-9330-98c14982d896" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 14 03:05:07.362524 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:07.362477    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:05:07.362958 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:07.362889    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:05:07.363501 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:07.363482    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:05:13.224772 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:13.224738    2574 generic.go:356] "Generic (PLEG): container finished" podID="21bae7db-ef7f-4897-8fbf-9f6923e01173" containerID="8f7f81d460e9c45202c3a0ae7645fab7f56d4bd47953a1349df4ac08f73ab5ce" exitCode=0
Sep 14 03:05:13.225209 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:13.224810    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz" event={"ID":"21bae7db-ef7f-4897-8fbf-9f6923e01173","Type":"ContainerDied","Data":"8f7f81d460e9c45202c3a0ae7645fab7f56d4bd47953a1349df4ac08f73ab5ce"}
Sep 14 03:05:13.225257 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:13.225212    2574 scope.go:117] "RemoveContainer" containerID="8f7f81d460e9c45202c3a0ae7645fab7f56d4bd47953a1349df4ac08f73ab5ce"
Sep 14 03:05:13.226390 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:13.226367    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:05:13.226453 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:13.226434    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:05:13.226487 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:13.226466    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:05:14.238686 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:14.238639    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz" event={"ID":"21bae7db-ef7f-4897-8fbf-9f6923e01173","Type":"ContainerStarted","Data":"abcce1f5dea8a79df4e8671a02aca9252dc7ced66f2a03709716fdcc0f36f756"}
Sep 14 03:05:14.814073 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:14.814036    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:05:14.814257 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:14.814216    2574 patch_prober.go:28] interesting pod/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 03:05:14.814307 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:14.814266    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz" podUID="21bae7db-ef7f-4897-8fbf-9f6923e01173" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 03:05:15.056255 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:15.056215    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:05:15.814437 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:15.814401    2574 patch_prober.go:28] interesting pod/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 03:05:15.814948 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:15.814464    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz" podUID="21bae7db-ef7f-4897-8fbf-9f6923e01173" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 03:05:15.950034 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:15.949988    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55"]
Sep 14 03:05:15.952843 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:15.952822    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55"
Sep 14 03:05:15.957267 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:15.957240    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4hf5p\""
Sep 14 03:05:15.957404 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:15.957271    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:05:15.970554 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:15.968701    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55"]
Sep 14 03:05:16.082271 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:16.082092    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9426161-a3c2-4406-8c88-322957575cde-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55\" (UID: \"f9426161-a3c2-4406-8c88-322957575cde\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55"
Sep 14 03:05:16.183805 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:16.182785    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9426161-a3c2-4406-8c88-322957575cde-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55\" (UID: \"f9426161-a3c2-4406-8c88-322957575cde\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55"
Sep 14 03:05:16.183805 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:16.183751    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9426161-a3c2-4406-8c88-322957575cde-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55\" (UID: \"f9426161-a3c2-4406-8c88-322957575cde\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55"
Sep 14 03:05:16.265315 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:16.265277    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55"
Sep 14 03:05:16.585406 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:16.585364    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55"]
Sep 14 03:05:16.601492 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:05:16.601441    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9426161_a3c2_4406_8c88_322957575cde.slice/crio-c712aeed6e40cd670aa93e158d5b0b77438b6a5e7abdcc373f94efffafadf892 WatchSource:0}: Error finding container c712aeed6e40cd670aa93e158d5b0b77438b6a5e7abdcc373f94efffafadf892: Status 404 returned error can't find the container with id c712aeed6e40cd670aa93e158d5b0b77438b6a5e7abdcc373f94efffafadf892
Sep 14 03:05:16.814242 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:16.814203    2574 patch_prober.go:28] interesting pod/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 03:05:16.814420 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:16.814260    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz" podUID="21bae7db-ef7f-4897-8fbf-9f6923e01173" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 03:05:17.255795 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:17.255757    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55" event={"ID":"f9426161-a3c2-4406-8c88-322957575cde","Type":"ContainerStarted","Data":"c712aeed6e40cd670aa93e158d5b0b77438b6a5e7abdcc373f94efffafadf892"}
Sep 14 03:05:17.814418 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:17.814380    2574 patch_prober.go:28] interesting pod/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 03:05:17.814619 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:17.814440    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz" podUID="21bae7db-ef7f-4897-8fbf-9f6923e01173" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 03:05:18.814338 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:18.814304    2574 patch_prober.go:28] interesting pod/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 03:05:18.814869 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:18.814366    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz" podUID="21bae7db-ef7f-4897-8fbf-9f6923e01173" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 03:05:19.292763 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:19.292695    2574 generic.go:356] "Generic (PLEG): container finished" podID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerID="fb9826212932a644196f4902ac373b14af3e92745d63d5b907e8f7ffd80892fc" exitCode=0
Sep 14 03:05:19.292763 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:19.292750    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f" event={"ID":"426574c6-8729-47fd-bc40-2c612dfcd7ac","Type":"ContainerDied","Data":"fb9826212932a644196f4902ac373b14af3e92745d63d5b907e8f7ffd80892fc"}
Sep 14 03:05:19.293275 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:19.293251    2574 scope.go:117] "RemoveContainer" containerID="fb9826212932a644196f4902ac373b14af3e92745d63d5b907e8f7ffd80892fc"
Sep 14 03:05:19.532685 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:19.532637    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:05:19.532869 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:19.532742    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:05:19.532869 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:19.532793    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:05:19.818931 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:19.818848    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:05:19.819341 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:19.819271    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:05:19.819985 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:19.819962    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:05:20.299662 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:20.299636    2574 generic.go:356] "Generic (PLEG): container finished" podID="f9426161-a3c2-4406-8c88-322957575cde" containerID="2a7a4d49158e8b5258009fa8e56acf314e1d094b9e897ca7660bb5dffd81a10f" exitCode=0
Sep 14 03:05:20.299780 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:20.299738    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55" event={"ID":"f9426161-a3c2-4406-8c88-322957575cde","Type":"ContainerDied","Data":"2a7a4d49158e8b5258009fa8e56acf314e1d094b9e897ca7660bb5dffd81a10f"}
Sep 14 03:05:20.301690 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:20.301658    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f" event={"ID":"426574c6-8729-47fd-bc40-2c612dfcd7ac","Type":"ContainerStarted","Data":"608bb2b1b030520dd7b5b992801084675c7aaae463ce919f612f1bc0e2b39f17"}
Sep 14 03:05:20.757274 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:20.757242    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:05:20.757451 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:20.757423    2574 patch_prober.go:28] interesting pod/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 14 03:05:20.757560 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:20.757477    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f" podUID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 14 03:05:21.449993 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:21.449973    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55"
Sep 14 03:05:21.562162 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:21.562083    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9426161-a3c2-4406-8c88-322957575cde-maas-api-service-ca\") pod \"f9426161-a3c2-4406-8c88-322957575cde\" (UID: \"f9426161-a3c2-4406-8c88-322957575cde\") "
Sep 14 03:05:21.562413 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:21.562390    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9426161-a3c2-4406-8c88-322957575cde-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f9426161-a3c2-4406-8c88-322957575cde" (UID: "f9426161-a3c2-4406-8c88-322957575cde"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:05:21.663717 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:21.663686    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9426161-a3c2-4406-8c88-322957575cde-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:05:21.756825 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:21.756792    2574 patch_prober.go:28] interesting pod/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 14 03:05:21.756979 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:21.756846    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f" podUID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 14 03:05:22.313123 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:22.313095    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55"
Sep 14 03:05:22.313123 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:22.313106    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55" event={"ID":"f9426161-a3c2-4406-8c88-322957575cde","Type":"ContainerDied","Data":"c712aeed6e40cd670aa93e158d5b0b77438b6a5e7abdcc373f94efffafadf892"}
Sep 14 03:05:22.313337 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:22.313153    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c712aeed6e40cd670aa93e158d5b0b77438b6a5e7abdcc373f94efffafadf892"
Sep 14 03:05:22.756855 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:22.756821    2574 patch_prober.go:28] interesting pod/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 14 03:05:22.757330 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:22.756881    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f" podUID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 14 03:05:23.757656 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:23.757620    2574 patch_prober.go:28] interesting pod/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 14 03:05:23.758041 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:23.757676    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f" podUID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 14 03:05:24.757012 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:24.756976    2574 patch_prober.go:28] interesting pod/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 14 03:05:24.757185 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:24.757041    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f" podUID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 14 03:05:25.757237 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:25.757200    2574 patch_prober.go:28] interesting pod/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 14 03:05:25.757635 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:25.757269    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f" podUID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 14 03:05:26.761013 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:26.760985    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:05:26.761415 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:26.761391    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:05:26.762159 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:26.762136    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:05:27.292419 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.292384    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"]
Sep 14 03:05:27.293001 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.292980    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9426161-a3c2-4406-8c88-322957575cde" containerName="revoke-keys"
Sep 14 03:05:27.293051 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.293006    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9426161-a3c2-4406-8c88-322957575cde" containerName="revoke-keys"
Sep 14 03:05:27.293100 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.293089    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9426161-a3c2-4406-8c88-322957575cde" containerName="revoke-keys"
Sep 14 03:05:27.295109 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.295090    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.298005 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.297975    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e467de\"/\"test-model-e467de-kserve-self-signed-certs\""
Sep 14 03:05:27.298205 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.298191    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e467de\"/\"openshift-service-ca.crt\""
Sep 14 03:05:27.298730 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.298711    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e467de\"/\"default-dockercfg-l567x\""
Sep 14 03:05:27.299241 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.299223    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e467de\"/\"kube-root-ca.crt\""
Sep 14 03:05:27.307655 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.307631    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"]
Sep 14 03:05:27.420007 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.419975    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-dshm\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.420007 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.420013    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c289e990-4e2d-4ce3-9a33-0226019a5c2b-tls-certs\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.420273 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.420072    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-model-cache\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.420273 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.420104    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b5lj\" (UniqueName: \"kubernetes.io/projected/c289e990-4e2d-4ce3-9a33-0226019a5c2b-kube-api-access-2b5lj\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.420273 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.420155    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-home\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.420273 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.420194    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-tmp-dir\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.521597 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.521551    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-dshm\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.521597 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.521594    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c289e990-4e2d-4ce3-9a33-0226019a5c2b-tls-certs\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.521863 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.521644    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-model-cache\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.521863 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.521683    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b5lj\" (UniqueName: \"kubernetes.io/projected/c289e990-4e2d-4ce3-9a33-0226019a5c2b-kube-api-access-2b5lj\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.521863 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.521744    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-home\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.521863 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.521790    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-tmp-dir\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.522203 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.522181    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-home\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.522271 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.522229    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-tmp-dir\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.522319 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.522296    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-model-cache\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.524080 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.524059    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-dshm\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.524615 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.524595    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c289e990-4e2d-4ce3-9a33-0226019a5c2b-tls-certs\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.535547 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.535508    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b5lj\" (UniqueName: \"kubernetes.io/projected/c289e990-4e2d-4ce3-9a33-0226019a5c2b-kube-api-access-2b5lj\") pod \"test-model-e467de-kserve-7b4c54dc9f-78lcx\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") " pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.606703 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.606609    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:27.896651 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:27.896622    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"]
Sep 14 03:05:27.897755 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:05:27.897731    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc289e990_4e2d_4ce3_9a33_0226019a5c2b.slice/crio-0e87aa33df0d8e3041e5b26cc6ee337bcbdd4c4aa173cd58eda946da76c47d95 WatchSource:0}: Error finding container 0e87aa33df0d8e3041e5b26cc6ee337bcbdd4c4aa173cd58eda946da76c47d95: Status 404 returned error can't find the container with id 0e87aa33df0d8e3041e5b26cc6ee337bcbdd4c4aa173cd58eda946da76c47d95
Sep 14 03:05:28.348424 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:28.348379    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx" event={"ID":"c289e990-4e2d-4ce3-9a33-0226019a5c2b","Type":"ContainerStarted","Data":"09a6301708fc238fba70e6a1f6e0d2e3d1e62ae2ad150b30baa56b15e1e6eb1e"}
Sep 14 03:05:28.348629 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:28.348436    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx" event={"ID":"c289e990-4e2d-4ce3-9a33-0226019a5c2b","Type":"ContainerStarted","Data":"0e87aa33df0d8e3041e5b26cc6ee337bcbdd4c4aa173cd58eda946da76c47d95"}
Sep 14 03:05:28.378225 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:28.378170    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx" podStartSLOduration=1.37815393 podStartE2EDuration="1.37815393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:05:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:05:28.376622702 +0000 UTC m=+874.689646523" watchObservedRunningTime="2026-09-14 03:05:28.37815393 +0000 UTC m=+874.691177766"
Sep 14 03:05:33.205910 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:33.205880    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55"]
Sep 14 03:05:33.215796 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:33.215767    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4b97c19-b9c55"]
Sep 14 03:05:33.385649 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:33.385619    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"]
Sep 14 03:05:33.385856 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:33.385826    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz" podUID="21bae7db-ef7f-4897-8fbf-9f6923e01173" containerName="istio-proxy" containerID="cri-o://abcce1f5dea8a79df4e8671a02aca9252dc7ced66f2a03709716fdcc0f36f756" gracePeriod=30
Sep 14 03:05:33.514222 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:33.514189    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:05:34.220066 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.219994    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9426161-a3c2-4406-8c88-322957575cde" path="/var/lib/kubelet/pods/f9426161-a3c2-4406-8c88-322957575cde/volumes"
Sep 14 03:05:34.673001 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.672965    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"]
Sep 14 03:05:34.675702 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.675686    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.679180 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.679156    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-07ec96b6-gw-openshift-default-dockercfg-tvfbk\""
Sep 14 03:05:34.687141 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.687118    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-workload-certs\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.687253 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.687146    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-podinfo\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.687253 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.687217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/504a990b-ee0e-42d0-a90a-5be36091fc3a-istiod-ca-cert\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.687253 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.687232    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jknh5\" (UniqueName: \"kubernetes.io/projected/504a990b-ee0e-42d0-a90a-5be36091fc3a-kube-api-access-jknh5\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.687397 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.687294    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-workload-socket\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.687397 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.687325    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-credential-socket\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.687397 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.687352    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-envoy\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.687397 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.687374    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-data\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.687540 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.687395    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-token\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.700175 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.700144    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"]
Sep 14 03:05:34.788305 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.788269    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/504a990b-ee0e-42d0-a90a-5be36091fc3a-istiod-ca-cert\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.788645 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.788619    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jknh5\" (UniqueName: \"kubernetes.io/projected/504a990b-ee0e-42d0-a90a-5be36091fc3a-kube-api-access-jknh5\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.788808 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.788791    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-workload-socket\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.788923 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.788904    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-credential-socket\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.789095 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.789071    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-envoy\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.789250 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.789107    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-data\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.789606 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.789548    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-credential-socket\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.789720 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.789684    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-token\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.789791 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.789755    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-data\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.789847 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.789800    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-workload-certs\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.789847 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.789828    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-podinfo\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.790320 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.790280    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-workload-certs\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.790985 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.790960    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-workload-socket\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.791105 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.791047    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/504a990b-ee0e-42d0-a90a-5be36091fc3a-istiod-ca-cert\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.792762 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.792383    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-envoy\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.792897 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.792875    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-podinfo\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.816471 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.816441    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-token\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.818038 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.818014    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jknh5\" (UniqueName: \"kubernetes.io/projected/504a990b-ee0e-42d0-a90a-5be36091fc3a-kube-api-access-jknh5\") pod \"e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") " pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:34.821802 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.821771    2574 patch_prober.go:28] interesting pod/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 14 03:05:34.821907 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.821848    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz" podUID="21bae7db-ef7f-4897-8fbf-9f6923e01173" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 03:05:34.986478 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:34.986441    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:35.283846 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:35.283747    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"]
Sep 14 03:05:35.289908 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:05:35.289764    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod504a990b_ee0e_42d0_a90a_5be36091fc3a.slice/crio-b3372f9f6cc7fc338ddabf72da0d5268d741c1c2f9b5fa2daf4f65fb6dc9abda WatchSource:0}: Error finding container b3372f9f6cc7fc338ddabf72da0d5268d741c1c2f9b5fa2daf4f65fb6dc9abda: Status 404 returned error can't find the container with id b3372f9f6cc7fc338ddabf72da0d5268d741c1c2f9b5fa2daf4f65fb6dc9abda
Sep 14 03:05:35.292623 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:35.292585    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:05:35.292708 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:35.292676    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:05:35.292749 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:35.292719    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:05:35.385212 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:35.385105    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb" event={"ID":"504a990b-ee0e-42d0-a90a-5be36091fc3a","Type":"ContainerStarted","Data":"3b111abffec6b73caacdfbc43eec87e9550ba8de76894bd527c9e809f4110df5"}
Sep 14 03:05:35.385212 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:35.385173    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb" event={"ID":"504a990b-ee0e-42d0-a90a-5be36091fc3a","Type":"ContainerStarted","Data":"b3372f9f6cc7fc338ddabf72da0d5268d741c1c2f9b5fa2daf4f65fb6dc9abda"}
Sep 14 03:05:35.416714 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:35.416615    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb" podStartSLOduration=1.416592131 podStartE2EDuration="1.416592131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:05:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:05:35.416508185 +0000 UTC m=+881.729532009" watchObservedRunningTime="2026-09-14 03:05:35.416592131 +0000 UTC m=+881.729615952"
Sep 14 03:05:35.987084 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:35.987050    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:36.991989 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:36.991959    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:37.396296 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:37.396215    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:37.397162 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:37.397143    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:37.607017 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:37.606975    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:37.607162 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:37.607060    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:37.619547 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:37.619491    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:38.414631 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.414607    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:05:38.652098 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.652025    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:05:38.731194 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731161    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-workload-certs\") pod \"21bae7db-ef7f-4897-8fbf-9f6923e01173\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") "
Sep 14 03:05:38.731194 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731194    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-envoy\") pod \"21bae7db-ef7f-4897-8fbf-9f6923e01173\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") "
Sep 14 03:05:38.731411 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731214    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-podinfo\") pod \"21bae7db-ef7f-4897-8fbf-9f6923e01173\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") "
Sep 14 03:05:38.731411 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731245    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-workload-socket\") pod \"21bae7db-ef7f-4897-8fbf-9f6923e01173\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") "
Sep 14 03:05:38.731411 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731277    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-token\") pod \"21bae7db-ef7f-4897-8fbf-9f6923e01173\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") "
Sep 14 03:05:38.731411 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731298    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4286j\" (UniqueName: \"kubernetes.io/projected/21bae7db-ef7f-4897-8fbf-9f6923e01173-kube-api-access-4286j\") pod \"21bae7db-ef7f-4897-8fbf-9f6923e01173\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") "
Sep 14 03:05:38.731411 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731350    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-data\") pod \"21bae7db-ef7f-4897-8fbf-9f6923e01173\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") "
Sep 14 03:05:38.731692 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731450    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-credential-socket\") pod \"21bae7db-ef7f-4897-8fbf-9f6923e01173\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") "
Sep 14 03:05:38.731692 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731459    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "21bae7db-ef7f-4897-8fbf-9f6923e01173" (UID: "21bae7db-ef7f-4897-8fbf-9f6923e01173"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:05:38.731692 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731489    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21bae7db-ef7f-4897-8fbf-9f6923e01173-istiod-ca-cert\") pod \"21bae7db-ef7f-4897-8fbf-9f6923e01173\" (UID: \"21bae7db-ef7f-4897-8fbf-9f6923e01173\") "
Sep 14 03:05:38.731692 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731589    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "21bae7db-ef7f-4897-8fbf-9f6923e01173" (UID: "21bae7db-ef7f-4897-8fbf-9f6923e01173"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:05:38.731692 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731678    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-data" (OuterVolumeSpecName: "istio-data") pod "21bae7db-ef7f-4897-8fbf-9f6923e01173" (UID: "21bae7db-ef7f-4897-8fbf-9f6923e01173"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:05:38.731953 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731847    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "21bae7db-ef7f-4897-8fbf-9f6923e01173" (UID: "21bae7db-ef7f-4897-8fbf-9f6923e01173"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:05:38.731953 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731877    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:05:38.731953 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731886    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21bae7db-ef7f-4897-8fbf-9f6923e01173-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "21bae7db-ef7f-4897-8fbf-9f6923e01173" (UID: "21bae7db-ef7f-4897-8fbf-9f6923e01173"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:05:38.731953 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731897    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:05:38.732317 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.731934    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:05:38.733791 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.733765    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "21bae7db-ef7f-4897-8fbf-9f6923e01173" (UID: "21bae7db-ef7f-4897-8fbf-9f6923e01173"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:05:38.733897 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.733801    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21bae7db-ef7f-4897-8fbf-9f6923e01173-kube-api-access-4286j" (OuterVolumeSpecName: "kube-api-access-4286j") pod "21bae7db-ef7f-4897-8fbf-9f6923e01173" (UID: "21bae7db-ef7f-4897-8fbf-9f6923e01173"). InnerVolumeSpecName "kube-api-access-4286j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:05:38.734035 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.734011    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "21bae7db-ef7f-4897-8fbf-9f6923e01173" (UID: "21bae7db-ef7f-4897-8fbf-9f6923e01173"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:05:38.734134 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.734013    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-token" (OuterVolumeSpecName: "istio-token") pod "21bae7db-ef7f-4897-8fbf-9f6923e01173" (UID: "21bae7db-ef7f-4897-8fbf-9f6923e01173"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:05:38.833269 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.833228    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:05:38.833269 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.833261    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21bae7db-ef7f-4897-8fbf-9f6923e01173-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:05:38.833269 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.833272    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:05:38.833269 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.833283    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:05:38.833604 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.833290    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21bae7db-ef7f-4897-8fbf-9f6923e01173-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:05:38.833604 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:38.833299    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4286j\" (UniqueName: \"kubernetes.io/projected/21bae7db-ef7f-4897-8fbf-9f6923e01173-kube-api-access-4286j\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:05:39.410572 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:39.410542    2574 generic.go:356] "Generic (PLEG): container finished" podID="21bae7db-ef7f-4897-8fbf-9f6923e01173" containerID="abcce1f5dea8a79df4e8671a02aca9252dc7ced66f2a03709716fdcc0f36f756" exitCode=0
Sep 14 03:05:39.410694 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:39.410586    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"
Sep 14 03:05:39.410694 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:39.410613    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz" event={"ID":"21bae7db-ef7f-4897-8fbf-9f6923e01173","Type":"ContainerDied","Data":"abcce1f5dea8a79df4e8671a02aca9252dc7ced66f2a03709716fdcc0f36f756"}
Sep 14 03:05:39.410694 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:39.410667    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz" event={"ID":"21bae7db-ef7f-4897-8fbf-9f6923e01173","Type":"ContainerDied","Data":"fb369417f1b153123f0fb1a768c950a958327428ddfc8c7cfa890cd0b5fa7b35"}
Sep 14 03:05:39.410694 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:39.410690    2574 scope.go:117] "RemoveContainer" containerID="abcce1f5dea8a79df4e8671a02aca9252dc7ced66f2a03709716fdcc0f36f756"
Sep 14 03:05:39.421153 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:39.421134    2574 scope.go:117] "RemoveContainer" containerID="8f7f81d460e9c45202c3a0ae7645fab7f56d4bd47953a1349df4ac08f73ab5ce"
Sep 14 03:05:39.429676 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:39.429657    2574 scope.go:117] "RemoveContainer" containerID="abcce1f5dea8a79df4e8671a02aca9252dc7ced66f2a03709716fdcc0f36f756"
Sep 14 03:05:39.429951 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:05:39.429934    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abcce1f5dea8a79df4e8671a02aca9252dc7ced66f2a03709716fdcc0f36f756\": container with ID starting with abcce1f5dea8a79df4e8671a02aca9252dc7ced66f2a03709716fdcc0f36f756 not found: ID does not exist" containerID="abcce1f5dea8a79df4e8671a02aca9252dc7ced66f2a03709716fdcc0f36f756"
Sep 14 03:05:39.429995 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:39.429961    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abcce1f5dea8a79df4e8671a02aca9252dc7ced66f2a03709716fdcc0f36f756"} err="failed to get container status \"abcce1f5dea8a79df4e8671a02aca9252dc7ced66f2a03709716fdcc0f36f756\": rpc error: code = NotFound desc = could not find container \"abcce1f5dea8a79df4e8671a02aca9252dc7ced66f2a03709716fdcc0f36f756\": container with ID starting with abcce1f5dea8a79df4e8671a02aca9252dc7ced66f2a03709716fdcc0f36f756 not found: ID does not exist"
Sep 14 03:05:39.429995 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:39.429978    2574 scope.go:117] "RemoveContainer" containerID="8f7f81d460e9c45202c3a0ae7645fab7f56d4bd47953a1349df4ac08f73ab5ce"
Sep 14 03:05:39.430228 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:05:39.430210    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f7f81d460e9c45202c3a0ae7645fab7f56d4bd47953a1349df4ac08f73ab5ce\": container with ID starting with 8f7f81d460e9c45202c3a0ae7645fab7f56d4bd47953a1349df4ac08f73ab5ce not found: ID does not exist" containerID="8f7f81d460e9c45202c3a0ae7645fab7f56d4bd47953a1349df4ac08f73ab5ce"
Sep 14 03:05:39.430277 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:39.430233    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f7f81d460e9c45202c3a0ae7645fab7f56d4bd47953a1349df4ac08f73ab5ce"} err="failed to get container status \"8f7f81d460e9c45202c3a0ae7645fab7f56d4bd47953a1349df4ac08f73ab5ce\": rpc error: code = NotFound desc = could not find container \"8f7f81d460e9c45202c3a0ae7645fab7f56d4bd47953a1349df4ac08f73ab5ce\": container with ID starting with 8f7f81d460e9c45202c3a0ae7645fab7f56d4bd47953a1349df4ac08f73ab5ce not found: ID does not exist"
Sep 14 03:05:39.442985 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:39.442962    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"]
Sep 14 03:05:39.451032 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:39.451008    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz"]
Sep 14 03:05:40.215565 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:40.215507    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21bae7db-ef7f-4897-8fbf-9f6923e01173" path="/var/lib/kubelet/pods/21bae7db-ef7f-4897-8fbf-9f6923e01173/volumes"
Sep 14 03:05:49.237173 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:49.237092    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:05:52.397928 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:52.397892    2574 patch_prober.go:28] interesting pod/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 03:05:52.398336 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:52.397950    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb" podUID="504a990b-ee0e-42d0-a90a-5be36091fc3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 03:05:52.487238 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:52.487206    2574 generic.go:356] "Generic (PLEG): container finished" podID="504a990b-ee0e-42d0-a90a-5be36091fc3a" containerID="3b111abffec6b73caacdfbc43eec87e9550ba8de76894bd527c9e809f4110df5" exitCode=0
Sep 14 03:05:52.487416 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:52.487282    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb" event={"ID":"504a990b-ee0e-42d0-a90a-5be36091fc3a","Type":"ContainerDied","Data":"3b111abffec6b73caacdfbc43eec87e9550ba8de76894bd527c9e809f4110df5"}
Sep 14 03:05:52.487732 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:52.487715    2574 scope.go:117] "RemoveContainer" containerID="3b111abffec6b73caacdfbc43eec87e9550ba8de76894bd527c9e809f4110df5"
Sep 14 03:05:52.488867 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:52.488813    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:05:52.488929 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:52.488890    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:05:52.488964 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:52.488926    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:05:53.495782 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:53.495749    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb" event={"ID":"504a990b-ee0e-42d0-a90a-5be36091fc3a","Type":"ContainerStarted","Data":"f9a6de68b5851a8044ea22a8c7140ff06bae277eaf43123cab0eb0dad057b56e"}
Sep 14 03:05:53.987727 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:53.987684    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:53.987938 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:53.987825    2574 patch_prober.go:28] interesting pod/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 03:05:53.987938 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:53.987881    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb" podUID="504a990b-ee0e-42d0-a90a-5be36091fc3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 03:05:54.156342 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.156312    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 03:05:54.158167 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.158144    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 03:05:54.162842 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.162822    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 03:05:54.164715 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.164696    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 03:05:54.402384 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.402285    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"]
Sep 14 03:05:54.402855 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.402832    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21bae7db-ef7f-4897-8fbf-9f6923e01173" containerName="istio-proxy"
Sep 14 03:05:54.402855 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.402855    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="21bae7db-ef7f-4897-8fbf-9f6923e01173" containerName="istio-proxy"
Sep 14 03:05:54.403063 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.402895    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21bae7db-ef7f-4897-8fbf-9f6923e01173" containerName="istio-proxy"
Sep 14 03:05:54.403063 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.402903    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="21bae7db-ef7f-4897-8fbf-9f6923e01173" containerName="istio-proxy"
Sep 14 03:05:54.403063 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.402986    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="21bae7db-ef7f-4897-8fbf-9f6923e01173" containerName="istio-proxy"
Sep 14 03:05:54.403063 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.403000    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="21bae7db-ef7f-4897-8fbf-9f6923e01173" containerName="istio-proxy"
Sep 14 03:05:54.407549 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.407500    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.410985 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.410872    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-bc1117\"/\"openshift-service-ca.crt\""
Sep 14 03:05:54.410985 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.410895    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-bc1117\"/\"kube-root-ca.crt\""
Sep 14 03:05:54.410985 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.410872    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-bc1117\"/\"default-dockercfg-kqlcr\""
Sep 14 03:05:54.411883 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.411867    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-bc1117\"/\"test-model-bc1117-kserve-self-signed-certs\""
Sep 14 03:05:54.419015 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.418988    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"]
Sep 14 03:05:54.486541 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.486041    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-dshm\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.486541 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.486124    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-tls-certs\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.486541 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.486195    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-home\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.486541 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.486264    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-model-cache\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.486541 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.486292    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plnxn\" (UniqueName: \"kubernetes.io/projected/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-kube-api-access-plnxn\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.486541 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.486320    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-tmp-dir\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.587919 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.587873    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-tls-certs\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.588404 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.587966    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-home\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.588404 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.588064    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-model-cache\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.588404 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.588105    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plnxn\" (UniqueName: \"kubernetes.io/projected/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-kube-api-access-plnxn\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.588404 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.588134    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-tmp-dir\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.588404 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.588217    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-dshm\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.589504 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.589450    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-tmp-dir\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.589504 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.589468    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-home\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.589716 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.589608    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-model-cache\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.591593 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.591563    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-dshm\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.592443 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.592419    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-tls-certs\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.598581 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.598548    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plnxn\" (UniqueName: \"kubernetes.io/projected/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-kube-api-access-plnxn\") pod \"test-model-bc1117-kserve-7866bc8bc5-49tfd\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") " pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.720300 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.720217    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:05:54.987160 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.987113    2574 patch_prober.go:28] interesting pod/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 03:05:54.987369 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:54.987199    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb" podUID="504a990b-ee0e-42d0-a90a-5be36091fc3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 03:05:55.029688 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:05:55.029655    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa7ed57f_56e7_4e86_a61b_1d19125a5ad2.slice/crio-e3e3497bfb3eeacf54da9e6bd4616ec11a4d344aa6d18ade4eee4a7d05c8c0d1 WatchSource:0}: Error finding container e3e3497bfb3eeacf54da9e6bd4616ec11a4d344aa6d18ade4eee4a7d05c8c0d1: Status 404 returned error can't find the container with id e3e3497bfb3eeacf54da9e6bd4616ec11a4d344aa6d18ade4eee4a7d05c8c0d1
Sep 14 03:05:55.029894 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:55.029872    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"]
Sep 14 03:05:55.509323 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:55.509285    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd" event={"ID":"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2","Type":"ContainerStarted","Data":"85e671c52781b64bc2912351561261257697110faf0f623ef75bb26cf878ed61"}
Sep 14 03:05:55.509323 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:55.509327    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd" event={"ID":"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2","Type":"ContainerStarted","Data":"e3e3497bfb3eeacf54da9e6bd4616ec11a4d344aa6d18ade4eee4a7d05c8c0d1"}
Sep 14 03:05:55.987782 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:55.987739    2574 patch_prober.go:28] interesting pod/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 03:05:55.988266 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:55.987807    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb" podUID="504a990b-ee0e-42d0-a90a-5be36091fc3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 03:05:56.987469 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:56.987428    2574 patch_prober.go:28] interesting pod/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 03:05:56.987813 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:56.987780    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb" podUID="504a990b-ee0e-42d0-a90a-5be36091fc3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 03:05:57.987409 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:57.987355    2574 patch_prober.go:28] interesting pod/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 03:05:57.987665 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:57.987433    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb" podUID="504a990b-ee0e-42d0-a90a-5be36091fc3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 03:05:58.990813 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:58.990773    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:58.991212 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:58.991070    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:58.991726 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:58.991711    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:05:59.020302 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:05:59.020236    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd" podStartSLOduration=5.020217963 podStartE2EDuration="5.020217963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:05:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:05:55.535361326 +0000 UTC m=+901.848385148" watchObservedRunningTime="2026-09-14 03:05:59.020217963 +0000 UTC m=+905.333241786"
Sep 14 03:06:04.721258 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:04.721222    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:06:04.721671 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:04.721276    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:06:04.734253 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:04.734228    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:06:05.572501 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:05.572476    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:06:06.841981 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:06.841948    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:06:19.408006 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:19.407974    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"]
Sep 14 03:06:19.408392 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:19.408213    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb" podUID="504a990b-ee0e-42d0-a90a-5be36091fc3a" containerName="istio-proxy" containerID="cri-o://f9a6de68b5851a8044ea22a8c7140ff06bae277eaf43123cab0eb0dad057b56e" gracePeriod=30
Sep 14 03:06:23.432813 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:23.432777    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:06:24.650731 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.650708    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:06:24.660003 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.659980    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-podinfo\") pod \"504a990b-ee0e-42d0-a90a-5be36091fc3a\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") "
Sep 14 03:06:24.660100 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660023    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-data\") pod \"504a990b-ee0e-42d0-a90a-5be36091fc3a\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") "
Sep 14 03:06:24.660100 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660059    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/504a990b-ee0e-42d0-a90a-5be36091fc3a-istiod-ca-cert\") pod \"504a990b-ee0e-42d0-a90a-5be36091fc3a\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") "
Sep 14 03:06:24.660209 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660111    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jknh5\" (UniqueName: \"kubernetes.io/projected/504a990b-ee0e-42d0-a90a-5be36091fc3a-kube-api-access-jknh5\") pod \"504a990b-ee0e-42d0-a90a-5be36091fc3a\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") "
Sep 14 03:06:24.660209 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660136    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-envoy\") pod \"504a990b-ee0e-42d0-a90a-5be36091fc3a\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") "
Sep 14 03:06:24.660209 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660157    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-token\") pod \"504a990b-ee0e-42d0-a90a-5be36091fc3a\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") "
Sep 14 03:06:24.660209 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660183    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-workload-certs\") pod \"504a990b-ee0e-42d0-a90a-5be36091fc3a\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") "
Sep 14 03:06:24.660394 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660216    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-workload-socket\") pod \"504a990b-ee0e-42d0-a90a-5be36091fc3a\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") "
Sep 14 03:06:24.660394 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660233    2574 generic.go:356] "Generic (PLEG): container finished" podID="504a990b-ee0e-42d0-a90a-5be36091fc3a" containerID="f9a6de68b5851a8044ea22a8c7140ff06bae277eaf43123cab0eb0dad057b56e" exitCode=0
Sep 14 03:06:24.660394 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660252    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-credential-socket\") pod \"504a990b-ee0e-42d0-a90a-5be36091fc3a\" (UID: \"504a990b-ee0e-42d0-a90a-5be36091fc3a\") "
Sep 14 03:06:24.660394 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660329    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb" event={"ID":"504a990b-ee0e-42d0-a90a-5be36091fc3a","Type":"ContainerDied","Data":"f9a6de68b5851a8044ea22a8c7140ff06bae277eaf43123cab0eb0dad057b56e"}
Sep 14 03:06:24.660394 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660335    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-data" (OuterVolumeSpecName: "istio-data") pod "504a990b-ee0e-42d0-a90a-5be36091fc3a" (UID: "504a990b-ee0e-42d0-a90a-5be36091fc3a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:06:24.660394 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660377    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"
Sep 14 03:06:24.660705 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660413    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb" event={"ID":"504a990b-ee0e-42d0-a90a-5be36091fc3a","Type":"ContainerDied","Data":"b3372f9f6cc7fc338ddabf72da0d5268d741c1c2f9b5fa2daf4f65fb6dc9abda"}
Sep 14 03:06:24.660705 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660445    2574 scope.go:117] "RemoveContainer" containerID="f9a6de68b5851a8044ea22a8c7140ff06bae277eaf43123cab0eb0dad057b56e"
Sep 14 03:06:24.660705 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660596    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:06:24.661212 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.660990    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "504a990b-ee0e-42d0-a90a-5be36091fc3a" (UID: "504a990b-ee0e-42d0-a90a-5be36091fc3a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:06:24.661212 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.661152    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/504a990b-ee0e-42d0-a90a-5be36091fc3a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "504a990b-ee0e-42d0-a90a-5be36091fc3a" (UID: "504a990b-ee0e-42d0-a90a-5be36091fc3a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:06:24.661558 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.661218    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "504a990b-ee0e-42d0-a90a-5be36091fc3a" (UID: "504a990b-ee0e-42d0-a90a-5be36091fc3a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:06:24.661558 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.661460    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "504a990b-ee0e-42d0-a90a-5be36091fc3a" (UID: "504a990b-ee0e-42d0-a90a-5be36091fc3a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:06:24.662830 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.662792    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "504a990b-ee0e-42d0-a90a-5be36091fc3a" (UID: "504a990b-ee0e-42d0-a90a-5be36091fc3a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:06:24.663310 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.663286    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "504a990b-ee0e-42d0-a90a-5be36091fc3a" (UID: "504a990b-ee0e-42d0-a90a-5be36091fc3a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:06:24.663434 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.663397    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/504a990b-ee0e-42d0-a90a-5be36091fc3a-kube-api-access-jknh5" (OuterVolumeSpecName: "kube-api-access-jknh5") pod "504a990b-ee0e-42d0-a90a-5be36091fc3a" (UID: "504a990b-ee0e-42d0-a90a-5be36091fc3a"). InnerVolumeSpecName "kube-api-access-jknh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:06:24.663723 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.663695    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-token" (OuterVolumeSpecName: "istio-token") pod "504a990b-ee0e-42d0-a90a-5be36091fc3a" (UID: "504a990b-ee0e-42d0-a90a-5be36091fc3a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:06:24.690146 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.690122    2574 scope.go:117] "RemoveContainer" containerID="3b111abffec6b73caacdfbc43eec87e9550ba8de76894bd527c9e809f4110df5"
Sep 14 03:06:24.698454 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.698436    2574 scope.go:117] "RemoveContainer" containerID="f9a6de68b5851a8044ea22a8c7140ff06bae277eaf43123cab0eb0dad057b56e"
Sep 14 03:06:24.698782 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:06:24.698762    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9a6de68b5851a8044ea22a8c7140ff06bae277eaf43123cab0eb0dad057b56e\": container with ID starting with f9a6de68b5851a8044ea22a8c7140ff06bae277eaf43123cab0eb0dad057b56e not found: ID does not exist" containerID="f9a6de68b5851a8044ea22a8c7140ff06bae277eaf43123cab0eb0dad057b56e"
Sep 14 03:06:24.698870 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.698797    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9a6de68b5851a8044ea22a8c7140ff06bae277eaf43123cab0eb0dad057b56e"} err="failed to get container status \"f9a6de68b5851a8044ea22a8c7140ff06bae277eaf43123cab0eb0dad057b56e\": rpc error: code = NotFound desc = could not find container \"f9a6de68b5851a8044ea22a8c7140ff06bae277eaf43123cab0eb0dad057b56e\": container with ID starting with f9a6de68b5851a8044ea22a8c7140ff06bae277eaf43123cab0eb0dad057b56e not found: ID does not exist"
Sep 14 03:06:24.698870 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.698831    2574 scope.go:117] "RemoveContainer" containerID="3b111abffec6b73caacdfbc43eec87e9550ba8de76894bd527c9e809f4110df5"
Sep 14 03:06:24.699145 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:06:24.699117    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b111abffec6b73caacdfbc43eec87e9550ba8de76894bd527c9e809f4110df5\": container with ID starting with 3b111abffec6b73caacdfbc43eec87e9550ba8de76894bd527c9e809f4110df5 not found: ID does not exist" containerID="3b111abffec6b73caacdfbc43eec87e9550ba8de76894bd527c9e809f4110df5"
Sep 14 03:06:24.699209 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.699156    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b111abffec6b73caacdfbc43eec87e9550ba8de76894bd527c9e809f4110df5"} err="failed to get container status \"3b111abffec6b73caacdfbc43eec87e9550ba8de76894bd527c9e809f4110df5\": rpc error: code = NotFound desc = could not find container \"3b111abffec6b73caacdfbc43eec87e9550ba8de76894bd527c9e809f4110df5\": container with ID starting with 3b111abffec6b73caacdfbc43eec87e9550ba8de76894bd527c9e809f4110df5 not found: ID does not exist"
Sep 14 03:06:24.761536 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.761487    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:06:24.761536 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.761509    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/504a990b-ee0e-42d0-a90a-5be36091fc3a-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:06:24.761536 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.761540    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jknh5\" (UniqueName: \"kubernetes.io/projected/504a990b-ee0e-42d0-a90a-5be36091fc3a-kube-api-access-jknh5\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:06:24.761704 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.761549    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:06:24.761704 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.761558    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/504a990b-ee0e-42d0-a90a-5be36091fc3a-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:06:24.761704 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.761565    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:06:24.761704 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.761572    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:06:24.761704 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.761580    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/504a990b-ee0e-42d0-a90a-5be36091fc3a-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:06:24.984401 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.984366    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"]
Sep 14 03:06:24.988863 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:24.988839    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb"]
Sep 14 03:06:26.216071 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:26.216031    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="504a990b-ee0e-42d0-a90a-5be36091fc3a" path="/var/lib/kubelet/pods/504a990b-ee0e-42d0-a90a-5be36091fc3a/volumes"
Sep 14 03:06:41.637464 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:41.637430    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:06:51.212802 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:51.212770    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4"]
Sep 14 03:06:51.213205 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:51.213115    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="504a990b-ee0e-42d0-a90a-5be36091fc3a" containerName="istio-proxy"
Sep 14 03:06:51.213205 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:51.213126    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="504a990b-ee0e-42d0-a90a-5be36091fc3a" containerName="istio-proxy"
Sep 14 03:06:51.213205 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:51.213202    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="504a990b-ee0e-42d0-a90a-5be36091fc3a" containerName="istio-proxy"
Sep 14 03:06:51.213313 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:51.213210    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="504a990b-ee0e-42d0-a90a-5be36091fc3a" containerName="istio-proxy"
Sep 14 03:06:51.216000 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:51.215982    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4"
Sep 14 03:06:51.218749 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:51.218728    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:06:51.218880 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:51.218827    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4hf5p\""
Sep 14 03:06:51.225411 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:51.225387    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4"]
Sep 14 03:06:51.284581 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:51.284557    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3f34790-9ce1-4e90-aeaf-34d17348bff2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4\" (UID: \"b3f34790-9ce1-4e90-aeaf-34d17348bff2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4"
Sep 14 03:06:51.385094 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:51.385066    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3f34790-9ce1-4e90-aeaf-34d17348bff2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4\" (UID: \"b3f34790-9ce1-4e90-aeaf-34d17348bff2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4"
Sep 14 03:06:51.385742 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:51.385725    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3f34790-9ce1-4e90-aeaf-34d17348bff2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4\" (UID: \"b3f34790-9ce1-4e90-aeaf-34d17348bff2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4"
Sep 14 03:06:51.527721 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:51.527704    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4"
Sep 14 03:06:51.801754 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:51.801731    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4"]
Sep 14 03:06:51.803257 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:06:51.803230    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3f34790_9ce1_4e90_aeaf_34d17348bff2.slice/crio-41a032f808ad6038b7663651ae3d83b99a32d42e8790e8cbb856ddf17cdb7c53 WatchSource:0}: Error finding container 41a032f808ad6038b7663651ae3d83b99a32d42e8790e8cbb856ddf17cdb7c53: Status 404 returned error can't find the container with id 41a032f808ad6038b7663651ae3d83b99a32d42e8790e8cbb856ddf17cdb7c53
Sep 14 03:06:52.803327 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:52.803293    2574 generic.go:356] "Generic (PLEG): container finished" podID="b3f34790-9ce1-4e90-aeaf-34d17348bff2" containerID="2ee7b5fd5e66cc5624a9be6b63d8b1bfafc5556915f9ab9f14402e715747b7d0" exitCode=0
Sep 14 03:06:52.803799 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:52.803351    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4" event={"ID":"b3f34790-9ce1-4e90-aeaf-34d17348bff2","Type":"ContainerDied","Data":"2ee7b5fd5e66cc5624a9be6b63d8b1bfafc5556915f9ab9f14402e715747b7d0"}
Sep 14 03:06:52.803799 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:52.803394    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4" event={"ID":"b3f34790-9ce1-4e90-aeaf-34d17348bff2","Type":"ContainerStarted","Data":"41a032f808ad6038b7663651ae3d83b99a32d42e8790e8cbb856ddf17cdb7c53"}
Sep 14 03:06:53.937154 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:53.937126    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4"
Sep 14 03:06:54.007817 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:54.007783    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3f34790-9ce1-4e90-aeaf-34d17348bff2-maas-api-service-ca\") pod \"b3f34790-9ce1-4e90-aeaf-34d17348bff2\" (UID: \"b3f34790-9ce1-4e90-aeaf-34d17348bff2\") "
Sep 14 03:06:54.008108 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:54.008088    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3f34790-9ce1-4e90-aeaf-34d17348bff2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b3f34790-9ce1-4e90-aeaf-34d17348bff2" (UID: "b3f34790-9ce1-4e90-aeaf-34d17348bff2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:06:54.109280 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:54.109219    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3f34790-9ce1-4e90-aeaf-34d17348bff2-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:06:54.815318 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:54.815288    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4"
Sep 14 03:06:54.815499 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:54.815325    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4" event={"ID":"b3f34790-9ce1-4e90-aeaf-34d17348bff2","Type":"ContainerDied","Data":"41a032f808ad6038b7663651ae3d83b99a32d42e8790e8cbb856ddf17cdb7c53"}
Sep 14 03:06:54.815499 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:54.815370    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41a032f808ad6038b7663651ae3d83b99a32d42e8790e8cbb856ddf17cdb7c53"
Sep 14 03:06:56.938121 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:56.938092    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:06:57.716657 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:57.716625    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4"]
Sep 14 03:06:57.719385 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:57.719362    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89bb5623-5wxk4"]
Sep 14 03:06:58.216429 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:06:58.216398    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3f34790-9ce1-4e90-aeaf-34d17348bff2" path="/var/lib/kubelet/pods/b3f34790-9ce1-4e90-aeaf-34d17348bff2/volumes"
Sep 14 03:07:11.145883 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:11.145842    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:07:13.745124 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:13.745091    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"]
Sep 14 03:07:14.654944 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:14.654915    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"]
Sep 14 03:07:14.781997 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:14.781963    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd"]
Sep 14 03:07:14.782386 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:14.782291    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="504a990b-ee0e-42d0-a90a-5be36091fc3a" containerName="istio-proxy"
Sep 14 03:07:14.782386 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:14.782302    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="504a990b-ee0e-42d0-a90a-5be36091fc3a" containerName="istio-proxy"
Sep 14 03:07:14.782386 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:14.782325    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3f34790-9ce1-4e90-aeaf-34d17348bff2" containerName="revoke-keys"
Sep 14 03:07:14.782386 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:14.782330    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3f34790-9ce1-4e90-aeaf-34d17348bff2" containerName="revoke-keys"
Sep 14 03:07:14.782567 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:14.782393    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3f34790-9ce1-4e90-aeaf-34d17348bff2" containerName="revoke-keys"
Sep 14 03:07:14.786674 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:14.786652    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd"
Sep 14 03:07:14.790012 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:14.789995    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:07:14.790118 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:14.790011    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4hf5p\""
Sep 14 03:07:14.795428 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:14.795405    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd"]
Sep 14 03:07:14.884366 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:14.884335    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/113d11f0-983d-495c-9ec6-040e7550961d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd\" (UID: \"113d11f0-983d-495c-9ec6-040e7550961d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd"
Sep 14 03:07:14.985193 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:14.985170    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/113d11f0-983d-495c-9ec6-040e7550961d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd\" (UID: \"113d11f0-983d-495c-9ec6-040e7550961d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd"
Sep 14 03:07:14.985800 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:14.985784    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/113d11f0-983d-495c-9ec6-040e7550961d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd\" (UID: \"113d11f0-983d-495c-9ec6-040e7550961d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd"
Sep 14 03:07:15.097969 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:15.097942    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd"
Sep 14 03:07:15.375857 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:15.375833    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd"]
Sep 14 03:07:15.377790 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:07:15.377768    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod113d11f0_983d_495c_9ec6_040e7550961d.slice/crio-5bfc077ac105c007bbe729123c1526f43253ef667a85f279a3c1d939a271481b WatchSource:0}: Error finding container 5bfc077ac105c007bbe729123c1526f43253ef667a85f279a3c1d939a271481b: Status 404 returned error can't find the container with id 5bfc077ac105c007bbe729123c1526f43253ef667a85f279a3c1d939a271481b
Sep 14 03:07:15.916742 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:15.916699    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd" event={"ID":"113d11f0-983d-495c-9ec6-040e7550961d","Type":"ContainerStarted","Data":"e32f69f8d47ba5a0b04f49004665753f1a983c827dde9383725e1ae185d73fa3"}
Sep 14 03:07:15.917122 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:15.916754    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd" event={"ID":"113d11f0-983d-495c-9ec6-040e7550961d","Type":"ContainerStarted","Data":"5bfc077ac105c007bbe729123c1526f43253ef667a85f279a3c1d939a271481b"}
Sep 14 03:07:15.939825 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:15.939727    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd" podStartSLOduration=1.9397137180000001 podStartE2EDuration="1.939713718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:07:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:07:15.938323548 +0000 UTC m=+982.251347365" watchObservedRunningTime="2026-09-14 03:07:15.939713718 +0000 UTC m=+982.252737537"
Sep 14 03:07:16.922943 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:16.922909    2574 generic.go:356] "Generic (PLEG): container finished" podID="113d11f0-983d-495c-9ec6-040e7550961d" containerID="e32f69f8d47ba5a0b04f49004665753f1a983c827dde9383725e1ae185d73fa3" exitCode=0
Sep 14 03:07:16.923342 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:16.922996    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd" event={"ID":"113d11f0-983d-495c-9ec6-040e7550961d","Type":"ContainerDied","Data":"e32f69f8d47ba5a0b04f49004665753f1a983c827dde9383725e1ae185d73fa3"}
Sep 14 03:07:18.080152 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:18.080130    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd"
Sep 14 03:07:18.106638 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:18.106617    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/113d11f0-983d-495c-9ec6-040e7550961d-maas-api-service-ca\") pod \"113d11f0-983d-495c-9ec6-040e7550961d\" (UID: \"113d11f0-983d-495c-9ec6-040e7550961d\") "
Sep 14 03:07:18.106957 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:18.106932    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/113d11f0-983d-495c-9ec6-040e7550961d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "113d11f0-983d-495c-9ec6-040e7550961d" (UID: "113d11f0-983d-495c-9ec6-040e7550961d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:18.207404 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:18.207319    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/113d11f0-983d-495c-9ec6-040e7550961d-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:18.934504 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:18.934464    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd" event={"ID":"113d11f0-983d-495c-9ec6-040e7550961d","Type":"ContainerDied","Data":"5bfc077ac105c007bbe729123c1526f43253ef667a85f279a3c1d939a271481b"}
Sep 14 03:07:18.934504 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:18.934506    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd"
Sep 14 03:07:18.934731 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:18.934538    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5bfc077ac105c007bbe729123c1526f43253ef667a85f279a3c1d939a271481b"
Sep 14 03:07:21.502470 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:21.502436    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-853daece-hdrf4"]
Sep 14 03:07:21.502874 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:21.502848    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113d11f0-983d-495c-9ec6-040e7550961d" containerName="revoke-keys"
Sep 14 03:07:21.502874 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:21.502860    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="113d11f0-983d-495c-9ec6-040e7550961d" containerName="revoke-keys"
Sep 14 03:07:21.502945 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:21.502934    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="113d11f0-983d-495c-9ec6-040e7550961d" containerName="revoke-keys"
Sep 14 03:07:21.505998 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:21.505983    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-853daece-hdrf4"
Sep 14 03:07:21.508547 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:21.508505    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4hf5p\""
Sep 14 03:07:21.508659 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:21.508545    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:07:21.512971 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:21.512916    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-853daece-hdrf4"]
Sep 14 03:07:21.531897 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:21.531872    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d7566dd-591d-4500-acc5-043006a79f71-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-853daece-hdrf4\" (UID: \"4d7566dd-591d-4500-acc5-043006a79f71\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-853daece-hdrf4"
Sep 14 03:07:21.632768 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:21.632739    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d7566dd-591d-4500-acc5-043006a79f71-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-853daece-hdrf4\" (UID: \"4d7566dd-591d-4500-acc5-043006a79f71\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-853daece-hdrf4"
Sep 14 03:07:21.633352 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:21.633332    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d7566dd-591d-4500-acc5-043006a79f71-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-853daece-hdrf4\" (UID: \"4d7566dd-591d-4500-acc5-043006a79f71\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-853daece-hdrf4"
Sep 14 03:07:21.818193 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:21.818108    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-853daece-hdrf4"
Sep 14 03:07:22.086950 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:22.086928    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-853daece-hdrf4"]
Sep 14 03:07:22.088643 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:07:22.088613    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d7566dd_591d_4500_acc5_043006a79f71.slice/crio-22cde2371954253619d9eb5620c46df0065c81246f638cb8c57e20fa36690e2d WatchSource:0}: Error finding container 22cde2371954253619d9eb5620c46df0065c81246f638cb8c57e20fa36690e2d: Status 404 returned error can't find the container with id 22cde2371954253619d9eb5620c46df0065c81246f638cb8c57e20fa36690e2d
Sep 14 03:07:22.956104 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:22.956068    2574 generic.go:356] "Generic (PLEG): container finished" podID="4d7566dd-591d-4500-acc5-043006a79f71" containerID="ada941ae22fabc81bfb1af81566bc9d18be2ad6f1a6b5f935e110304d565e62b" exitCode=0
Sep 14 03:07:22.956451 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:22.956146    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-853daece-hdrf4" event={"ID":"4d7566dd-591d-4500-acc5-043006a79f71","Type":"ContainerDied","Data":"ada941ae22fabc81bfb1af81566bc9d18be2ad6f1a6b5f935e110304d565e62b"}
Sep 14 03:07:22.956451 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:22.956203    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-853daece-hdrf4" event={"ID":"4d7566dd-591d-4500-acc5-043006a79f71","Type":"ContainerStarted","Data":"22cde2371954253619d9eb5620c46df0065c81246f638cb8c57e20fa36690e2d"}
Sep 14 03:07:24.094335 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:24.094310    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-853daece-hdrf4"
Sep 14 03:07:24.153010 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:24.152987    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d7566dd-591d-4500-acc5-043006a79f71-maas-api-service-ca\") pod \"4d7566dd-591d-4500-acc5-043006a79f71\" (UID: \"4d7566dd-591d-4500-acc5-043006a79f71\") "
Sep 14 03:07:24.153333 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:24.153309    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d7566dd-591d-4500-acc5-043006a79f71-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d7566dd-591d-4500-acc5-043006a79f71" (UID: "4d7566dd-591d-4500-acc5-043006a79f71"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:24.254286 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:24.254264    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d7566dd-591d-4500-acc5-043006a79f71-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:24.969259 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:24.969230    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-853daece-hdrf4"
Sep 14 03:07:24.969446 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:24.969273    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-853daece-hdrf4" event={"ID":"4d7566dd-591d-4500-acc5-043006a79f71","Type":"ContainerDied","Data":"22cde2371954253619d9eb5620c46df0065c81246f638cb8c57e20fa36690e2d"}
Sep 14 03:07:24.969446 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:24.969320    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22cde2371954253619d9eb5620c46df0065c81246f638cb8c57e20fa36690e2d"
Sep 14 03:07:25.808700 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:25.808669    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd"]
Sep 14 03:07:25.810368 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:25.810332    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e467de-nb7kd"]
Sep 14 03:07:26.215783 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:26.215710    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="113d11f0-983d-495c-9ec6-040e7550961d" path="/var/lib/kubelet/pods/113d11f0-983d-495c-9ec6-040e7550961d/volumes"
Sep 14 03:07:28.764211 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:28.764153    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx" podUID="c289e990-4e2d-4ce3-9a33-0226019a5c2b" containerName="main" containerID="cri-o://09a6301708fc238fba70e6a1f6e0d2e3d1e62ae2ad150b30baa56b15e1e6eb1e" gracePeriod=45
Sep 14 03:07:28.990109 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:28.990074    2574 generic.go:356] "Generic (PLEG): container finished" podID="c289e990-4e2d-4ce3-9a33-0226019a5c2b" containerID="09a6301708fc238fba70e6a1f6e0d2e3d1e62ae2ad150b30baa56b15e1e6eb1e" exitCode=0
Sep 14 03:07:28.990235 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:28.990140    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx" event={"ID":"c289e990-4e2d-4ce3-9a33-0226019a5c2b","Type":"ContainerDied","Data":"09a6301708fc238fba70e6a1f6e0d2e3d1e62ae2ad150b30baa56b15e1e6eb1e"}
Sep 14 03:07:29.017568 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.017503    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:07:29.096481 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.096456    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c289e990-4e2d-4ce3-9a33-0226019a5c2b-tls-certs\") pod \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") "
Sep 14 03:07:29.096638 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.096485    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2b5lj\" (UniqueName: \"kubernetes.io/projected/c289e990-4e2d-4ce3-9a33-0226019a5c2b-kube-api-access-2b5lj\") pod \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") "
Sep 14 03:07:29.096638 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.096504    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-dshm\") pod \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") "
Sep 14 03:07:29.096638 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.096550    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-model-cache\") pod \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") "
Sep 14 03:07:29.096638 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.096605    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-home\") pod \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") "
Sep 14 03:07:29.096847 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.096656    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-tmp-dir\") pod \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\" (UID: \"c289e990-4e2d-4ce3-9a33-0226019a5c2b\") "
Sep 14 03:07:29.096910 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.096880    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-model-cache" (OuterVolumeSpecName: "model-cache") pod "c289e990-4e2d-4ce3-9a33-0226019a5c2b" (UID: "c289e990-4e2d-4ce3-9a33-0226019a5c2b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:29.096948 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.096903    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-home" (OuterVolumeSpecName: "home") pod "c289e990-4e2d-4ce3-9a33-0226019a5c2b" (UID: "c289e990-4e2d-4ce3-9a33-0226019a5c2b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:29.097075 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.097041    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c289e990-4e2d-4ce3-9a33-0226019a5c2b" (UID: "c289e990-4e2d-4ce3-9a33-0226019a5c2b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:29.098905 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.098874    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-dshm" (OuterVolumeSpecName: "dshm") pod "c289e990-4e2d-4ce3-9a33-0226019a5c2b" (UID: "c289e990-4e2d-4ce3-9a33-0226019a5c2b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:29.098986 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.098953    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c289e990-4e2d-4ce3-9a33-0226019a5c2b-kube-api-access-2b5lj" (OuterVolumeSpecName: "kube-api-access-2b5lj") pod "c289e990-4e2d-4ce3-9a33-0226019a5c2b" (UID: "c289e990-4e2d-4ce3-9a33-0226019a5c2b"). InnerVolumeSpecName "kube-api-access-2b5lj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:29.098986 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.098974    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c289e990-4e2d-4ce3-9a33-0226019a5c2b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c289e990-4e2d-4ce3-9a33-0226019a5c2b" (UID: "c289e990-4e2d-4ce3-9a33-0226019a5c2b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:07:29.144227 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.144206    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:07:29.198300 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.198271    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c289e990-4e2d-4ce3-9a33-0226019a5c2b-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:29.198532 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.198492    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2b5lj\" (UniqueName: \"kubernetes.io/projected/c289e990-4e2d-4ce3-9a33-0226019a5c2b-kube-api-access-2b5lj\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:29.198676 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.198536    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:29.198676 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.198552    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:29.198676 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.198565    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:29.198676 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.198577    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c289e990-4e2d-4ce3-9a33-0226019a5c2b-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:29.674322 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.674290    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd" podUID="aa7ed57f-56e7-4e86-a61b-1d19125a5ad2" containerName="main" containerID="cri-o://85e671c52781b64bc2912351561261257697110faf0f623ef75bb26cf878ed61" gracePeriod=45
Sep 14 03:07:29.910984 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.910960    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:07:29.995911 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.995877    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx" event={"ID":"c289e990-4e2d-4ce3-9a33-0226019a5c2b","Type":"ContainerDied","Data":"0e87aa33df0d8e3041e5b26cc6ee337bcbdd4c4aa173cd58eda946da76c47d95"}
Sep 14 03:07:29.996053 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.995930    2574 scope.go:117] "RemoveContainer" containerID="09a6301708fc238fba70e6a1f6e0d2e3d1e62ae2ad150b30baa56b15e1e6eb1e"
Sep 14 03:07:29.996053 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.995887    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"
Sep 14 03:07:29.997350 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.997331    2574 generic.go:356] "Generic (PLEG): container finished" podID="aa7ed57f-56e7-4e86-a61b-1d19125a5ad2" containerID="85e671c52781b64bc2912351561261257697110faf0f623ef75bb26cf878ed61" exitCode=0
Sep 14 03:07:29.997446 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.997369    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"
Sep 14 03:07:29.997446 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.997408    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd" event={"ID":"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2","Type":"ContainerDied","Data":"85e671c52781b64bc2912351561261257697110faf0f623ef75bb26cf878ed61"}
Sep 14 03:07:29.997541 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:29.997448    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd" event={"ID":"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2","Type":"ContainerDied","Data":"e3e3497bfb3eeacf54da9e6bd4616ec11a4d344aa6d18ade4eee4a7d05c8c0d1"}
Sep 14 03:07:30.004634 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.004617    2574 scope.go:117] "RemoveContainer" containerID="85e671c52781b64bc2912351561261257697110faf0f623ef75bb26cf878ed61"
Sep 14 03:07:30.006716 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.006701    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-home\") pod \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") "
Sep 14 03:07:30.006815 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.006784    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-model-cache\") pod \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") "
Sep 14 03:07:30.006895 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.006855    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-tls-certs\") pod \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") "
Sep 14 03:07:30.006946 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.006911    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-dshm\") pod \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") "
Sep 14 03:07:30.007000 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.006942    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-tmp-dir\") pod \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") "
Sep 14 03:07:30.007000 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.006945    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-home" (OuterVolumeSpecName: "home") pod "aa7ed57f-56e7-4e86-a61b-1d19125a5ad2" (UID: "aa7ed57f-56e7-4e86-a61b-1d19125a5ad2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:30.007000 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.006977    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plnxn\" (UniqueName: \"kubernetes.io/projected/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-kube-api-access-plnxn\") pod \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\" (UID: \"aa7ed57f-56e7-4e86-a61b-1d19125a5ad2\") "
Sep 14 03:07:30.007142 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.007023    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-model-cache" (OuterVolumeSpecName: "model-cache") pod "aa7ed57f-56e7-4e86-a61b-1d19125a5ad2" (UID: "aa7ed57f-56e7-4e86-a61b-1d19125a5ad2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:30.007194 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.007163    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aa7ed57f-56e7-4e86-a61b-1d19125a5ad2" (UID: "aa7ed57f-56e7-4e86-a61b-1d19125a5ad2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:30.007271 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.007249    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:30.007380 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.007276    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:30.007380 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.007293    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:30.009160 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.009136    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aa7ed57f-56e7-4e86-a61b-1d19125a5ad2" (UID: "aa7ed57f-56e7-4e86-a61b-1d19125a5ad2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:07:30.009251 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.009173    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-dshm" (OuterVolumeSpecName: "dshm") pod "aa7ed57f-56e7-4e86-a61b-1d19125a5ad2" (UID: "aa7ed57f-56e7-4e86-a61b-1d19125a5ad2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:30.009251 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.009195    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-kube-api-access-plnxn" (OuterVolumeSpecName: "kube-api-access-plnxn") pod "aa7ed57f-56e7-4e86-a61b-1d19125a5ad2" (UID: "aa7ed57f-56e7-4e86-a61b-1d19125a5ad2"). InnerVolumeSpecName "kube-api-access-plnxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:30.013473 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.013458    2574 scope.go:117] "RemoveContainer" containerID="85e671c52781b64bc2912351561261257697110faf0f623ef75bb26cf878ed61"
Sep 14 03:07:30.013770 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:07:30.013748    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85e671c52781b64bc2912351561261257697110faf0f623ef75bb26cf878ed61\": container with ID starting with 85e671c52781b64bc2912351561261257697110faf0f623ef75bb26cf878ed61 not found: ID does not exist" containerID="85e671c52781b64bc2912351561261257697110faf0f623ef75bb26cf878ed61"
Sep 14 03:07:30.013866 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.013776    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85e671c52781b64bc2912351561261257697110faf0f623ef75bb26cf878ed61"} err="failed to get container status \"85e671c52781b64bc2912351561261257697110faf0f623ef75bb26cf878ed61\": rpc error: code = NotFound desc = could not find container \"85e671c52781b64bc2912351561261257697110faf0f623ef75bb26cf878ed61\": container with ID starting with 85e671c52781b64bc2912351561261257697110faf0f623ef75bb26cf878ed61 not found: ID does not exist"
Sep 14 03:07:30.037693 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.037673    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"]
Sep 14 03:07:30.045521 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.045491    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e467de/test-model-e467de-kserve-7b4c54dc9f-78lcx"]
Sep 14 03:07:30.107779 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.107757    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:30.107779 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.107776    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:30.107888 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.107785    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plnxn\" (UniqueName: \"kubernetes.io/projected/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2-kube-api-access-plnxn\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:30.215347 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.215325    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c289e990-4e2d-4ce3-9a33-0226019a5c2b" path="/var/lib/kubelet/pods/c289e990-4e2d-4ce3-9a33-0226019a5c2b/volumes"
Sep 14 03:07:30.313871 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.313820    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"]
Sep 14 03:07:30.320213 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:30.320189    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-bc1117/test-model-bc1117-kserve-7866bc8bc5-49tfd"]
Sep 14 03:07:32.215802 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:32.215776    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa7ed57f-56e7-4e86-a61b-1d19125a5ad2" path="/var/lib/kubelet/pods/aa7ed57f-56e7-4e86-a61b-1d19125a5ad2/volumes"
Sep 14 03:07:32.310416 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:32.310387    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-853daece-hdrf4"]
Sep 14 03:07:32.313928 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:32.313895    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-853daece-hdrf4"]
Sep 14 03:07:34.216215 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:34.216183    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d7566dd-591d-4500-acc5-043006a79f71" path="/var/lib/kubelet/pods/4d7566dd-591d-4500-acc5-043006a79f71/volumes"
Sep 14 03:07:34.686136 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:34.686100    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"]
Sep 14 03:07:34.686356 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:34.686329    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4" podUID="7edbe9e7-db64-421a-9330-98c14982d896" containerName="istio-proxy" containerID="cri-o://1c147cc95563049040f23626a8c374e11ed9c8c73a91d5196b924b97f9a2fb57" gracePeriod=30
Sep 14 03:07:35.053205 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.053176    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs"]
Sep 14 03:07:35.053540 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.053498    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa7ed57f-56e7-4e86-a61b-1d19125a5ad2" containerName="main"
Sep 14 03:07:35.053540 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.053525    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa7ed57f-56e7-4e86-a61b-1d19125a5ad2" containerName="main"
Sep 14 03:07:35.053633 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.053537    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c289e990-4e2d-4ce3-9a33-0226019a5c2b" containerName="main"
Sep 14 03:07:35.053633 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.053553    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c289e990-4e2d-4ce3-9a33-0226019a5c2b" containerName="main"
Sep 14 03:07:35.053633 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.053567    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d7566dd-591d-4500-acc5-043006a79f71" containerName="revoke-keys"
Sep 14 03:07:35.053633 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.053573    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d7566dd-591d-4500-acc5-043006a79f71" containerName="revoke-keys"
Sep 14 03:07:35.053633 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.053623    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c289e990-4e2d-4ce3-9a33-0226019a5c2b" containerName="main"
Sep 14 03:07:35.053633 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.053632    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa7ed57f-56e7-4e86-a61b-1d19125a5ad2" containerName="main"
Sep 14 03:07:35.053633 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.053640    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d7566dd-591d-4500-acc5-043006a79f71" containerName="revoke-keys"
Sep 14 03:07:35.057864 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.057847    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs"
Sep 14 03:07:35.061644 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.061621    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:07:35.061644 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.061637    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4hf5p\""
Sep 14 03:07:35.068001 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.067979    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs"]
Sep 14 03:07:35.161076 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.161049    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e73d3f-4423-4f61-9b31-0ef4787f929c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-bc1117-874fs\" (UID: \"14e73d3f-4423-4f61-9b31-0ef4787f929c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs"
Sep 14 03:07:35.262455 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.262427    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e73d3f-4423-4f61-9b31-0ef4787f929c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-bc1117-874fs\" (UID: \"14e73d3f-4423-4f61-9b31-0ef4787f929c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs"
Sep 14 03:07:35.263004 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.262986    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e73d3f-4423-4f61-9b31-0ef4787f929c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-bc1117-874fs\" (UID: \"14e73d3f-4423-4f61-9b31-0ef4787f929c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs"
Sep 14 03:07:35.368492 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.368411    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs"
Sep 14 03:07:35.637210 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:35.637189    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs"]
Sep 14 03:07:35.638975 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:07:35.638950    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14e73d3f_4423_4f61_9b31_0ef4787f929c.slice/crio-443435d7498d357306e77b36b5d6a0f34a3c2677cd38e0a7599f072aac6f0f97 WatchSource:0}: Error finding container 443435d7498d357306e77b36b5d6a0f34a3c2677cd38e0a7599f072aac6f0f97: Status 404 returned error can't find the container with id 443435d7498d357306e77b36b5d6a0f34a3c2677cd38e0a7599f072aac6f0f97
Sep 14 03:07:36.037912 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:36.037882    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs" event={"ID":"14e73d3f-4423-4f61-9b31-0ef4787f929c","Type":"ContainerStarted","Data":"a6f2713ebb6786284413bc68ed307a67954374af912b9f22b8ebe0efb9eb735c"}
Sep 14 03:07:36.038049 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:36.037930    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs" event={"ID":"14e73d3f-4423-4f61-9b31-0ef4787f929c","Type":"ContainerStarted","Data":"443435d7498d357306e77b36b5d6a0f34a3c2677cd38e0a7599f072aac6f0f97"}
Sep 14 03:07:36.056266 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:36.056228    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs" podStartSLOduration=1.056213849 podStartE2EDuration="1.056213849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:07:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:07:36.05482369 +0000 UTC m=+1002.367847509" watchObservedRunningTime="2026-09-14 03:07:36.056213849 +0000 UTC m=+1002.369237668"
Sep 14 03:07:37.045164 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:37.045133    2574 generic.go:356] "Generic (PLEG): container finished" podID="14e73d3f-4423-4f61-9b31-0ef4787f929c" containerID="a6f2713ebb6786284413bc68ed307a67954374af912b9f22b8ebe0efb9eb735c" exitCode=0
Sep 14 03:07:37.045571 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:37.045210    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs" event={"ID":"14e73d3f-4423-4f61-9b31-0ef4787f929c","Type":"ContainerDied","Data":"a6f2713ebb6786284413bc68ed307a67954374af912b9f22b8ebe0efb9eb735c"}
Sep 14 03:07:37.364797 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:37.364718    2574 patch_prober.go:28] interesting pod/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 03:07:37.364797 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:37.364774    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4" podUID="7edbe9e7-db64-421a-9330-98c14982d896" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 03:07:38.179730 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:38.179707    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs"
Sep 14 03:07:38.289371 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:38.289340    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e73d3f-4423-4f61-9b31-0ef4787f929c-maas-api-service-ca\") pod \"14e73d3f-4423-4f61-9b31-0ef4787f929c\" (UID: \"14e73d3f-4423-4f61-9b31-0ef4787f929c\") "
Sep 14 03:07:38.289764 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:38.289739    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14e73d3f-4423-4f61-9b31-0ef4787f929c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "14e73d3f-4423-4f61-9b31-0ef4787f929c" (UID: "14e73d3f-4423-4f61-9b31-0ef4787f929c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:38.390809 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:38.390733    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e73d3f-4423-4f61-9b31-0ef4787f929c-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:39.056821 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:39.056789    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs"
Sep 14 03:07:39.056821 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:39.056796    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs" event={"ID":"14e73d3f-4423-4f61-9b31-0ef4787f929c","Type":"ContainerDied","Data":"443435d7498d357306e77b36b5d6a0f34a3c2677cd38e0a7599f072aac6f0f97"}
Sep 14 03:07:39.057017 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:39.056847    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="443435d7498d357306e77b36b5d6a0f34a3c2677cd38e0a7599f072aac6f0f97"
Sep 14 03:07:39.932182 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:39.932156    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:07:40.003732 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.003703    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7edbe9e7-db64-421a-9330-98c14982d896-istiod-ca-cert\") pod \"7edbe9e7-db64-421a-9330-98c14982d896\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") "
Sep 14 03:07:40.003881 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.003766    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7edbe9e7-db64-421a-9330-98c14982d896-istio-token\") pod \"7edbe9e7-db64-421a-9330-98c14982d896\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") "
Sep 14 03:07:40.003881 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.003786    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-istio-envoy\") pod \"7edbe9e7-db64-421a-9330-98c14982d896\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") "
Sep 14 03:07:40.003881 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.003804    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gjjk\" (UniqueName: \"kubernetes.io/projected/7edbe9e7-db64-421a-9330-98c14982d896-kube-api-access-7gjjk\") pod \"7edbe9e7-db64-421a-9330-98c14982d896\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") "
Sep 14 03:07:40.003881 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.003819    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-istio-data\") pod \"7edbe9e7-db64-421a-9330-98c14982d896\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") "
Sep 14 03:07:40.003881 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.003844    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-credential-socket\") pod \"7edbe9e7-db64-421a-9330-98c14982d896\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") "
Sep 14 03:07:40.003881 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.003876    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-workload-socket\") pod \"7edbe9e7-db64-421a-9330-98c14982d896\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") "
Sep 14 03:07:40.004147 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.003912    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7edbe9e7-db64-421a-9330-98c14982d896-istio-podinfo\") pod \"7edbe9e7-db64-421a-9330-98c14982d896\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") "
Sep 14 03:07:40.004147 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.003939    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-workload-certs\") pod \"7edbe9e7-db64-421a-9330-98c14982d896\" (UID: \"7edbe9e7-db64-421a-9330-98c14982d896\") "
Sep 14 03:07:40.004248 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.004176    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7edbe9e7-db64-421a-9330-98c14982d896" (UID: "7edbe9e7-db64-421a-9330-98c14982d896"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:40.004248 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.004199    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7edbe9e7-db64-421a-9330-98c14982d896-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7edbe9e7-db64-421a-9330-98c14982d896" (UID: "7edbe9e7-db64-421a-9330-98c14982d896"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:40.004248 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.004200    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7edbe9e7-db64-421a-9330-98c14982d896" (UID: "7edbe9e7-db64-421a-9330-98c14982d896"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:40.004402 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.004289    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-istio-data" (OuterVolumeSpecName: "istio-data") pod "7edbe9e7-db64-421a-9330-98c14982d896" (UID: "7edbe9e7-db64-421a-9330-98c14982d896"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:40.004457 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.004429    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7edbe9e7-db64-421a-9330-98c14982d896" (UID: "7edbe9e7-db64-421a-9330-98c14982d896"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:40.006133 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.006108    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7edbe9e7-db64-421a-9330-98c14982d896-istio-token" (OuterVolumeSpecName: "istio-token") pod "7edbe9e7-db64-421a-9330-98c14982d896" (UID: "7edbe9e7-db64-421a-9330-98c14982d896"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:40.006268 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.006223    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7edbe9e7-db64-421a-9330-98c14982d896" (UID: "7edbe9e7-db64-421a-9330-98c14982d896"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:40.006426 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.006398    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7edbe9e7-db64-421a-9330-98c14982d896-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7edbe9e7-db64-421a-9330-98c14982d896" (UID: "7edbe9e7-db64-421a-9330-98c14982d896"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:07:40.006571 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.006555    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7edbe9e7-db64-421a-9330-98c14982d896-kube-api-access-7gjjk" (OuterVolumeSpecName: "kube-api-access-7gjjk") pod "7edbe9e7-db64-421a-9330-98c14982d896" (UID: "7edbe9e7-db64-421a-9330-98c14982d896"). InnerVolumeSpecName "kube-api-access-7gjjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:40.062889 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.062860    2574 generic.go:356] "Generic (PLEG): container finished" podID="7edbe9e7-db64-421a-9330-98c14982d896" containerID="1c147cc95563049040f23626a8c374e11ed9c8c73a91d5196b924b97f9a2fb57" exitCode=0
Sep 14 03:07:40.062996 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.062909    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"
Sep 14 03:07:40.062996 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.062949    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4" event={"ID":"7edbe9e7-db64-421a-9330-98c14982d896","Type":"ContainerDied","Data":"1c147cc95563049040f23626a8c374e11ed9c8c73a91d5196b924b97f9a2fb57"}
Sep 14 03:07:40.063094 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.063007    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4" event={"ID":"7edbe9e7-db64-421a-9330-98c14982d896","Type":"ContainerDied","Data":"d1be016b6391ff3a0153cfba6df011984d1c4800c446b44fcf094836562a07fc"}
Sep 14 03:07:40.063094 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.063023    2574 scope.go:117] "RemoveContainer" containerID="1c147cc95563049040f23626a8c374e11ed9c8c73a91d5196b924b97f9a2fb57"
Sep 14 03:07:40.072371 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.072355    2574 scope.go:117] "RemoveContainer" containerID="11654c6a755fd96a610bc8f62740d4275d0560427786092eadaa457d0a53e64b"
Sep 14 03:07:40.079972 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.079956    2574 scope.go:117] "RemoveContainer" containerID="1c147cc95563049040f23626a8c374e11ed9c8c73a91d5196b924b97f9a2fb57"
Sep 14 03:07:40.080234 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:07:40.080213    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c147cc95563049040f23626a8c374e11ed9c8c73a91d5196b924b97f9a2fb57\": container with ID starting with 1c147cc95563049040f23626a8c374e11ed9c8c73a91d5196b924b97f9a2fb57 not found: ID does not exist" containerID="1c147cc95563049040f23626a8c374e11ed9c8c73a91d5196b924b97f9a2fb57"
Sep 14 03:07:40.080310 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.080243    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c147cc95563049040f23626a8c374e11ed9c8c73a91d5196b924b97f9a2fb57"} err="failed to get container status \"1c147cc95563049040f23626a8c374e11ed9c8c73a91d5196b924b97f9a2fb57\": rpc error: code = NotFound desc = could not find container \"1c147cc95563049040f23626a8c374e11ed9c8c73a91d5196b924b97f9a2fb57\": container with ID starting with 1c147cc95563049040f23626a8c374e11ed9c8c73a91d5196b924b97f9a2fb57 not found: ID does not exist"
Sep 14 03:07:40.080310 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.080265    2574 scope.go:117] "RemoveContainer" containerID="11654c6a755fd96a610bc8f62740d4275d0560427786092eadaa457d0a53e64b"
Sep 14 03:07:40.080500 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:07:40.080485    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11654c6a755fd96a610bc8f62740d4275d0560427786092eadaa457d0a53e64b\": container with ID starting with 11654c6a755fd96a610bc8f62740d4275d0560427786092eadaa457d0a53e64b not found: ID does not exist" containerID="11654c6a755fd96a610bc8f62740d4275d0560427786092eadaa457d0a53e64b"
Sep 14 03:07:40.080589 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.080506    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11654c6a755fd96a610bc8f62740d4275d0560427786092eadaa457d0a53e64b"} err="failed to get container status \"11654c6a755fd96a610bc8f62740d4275d0560427786092eadaa457d0a53e64b\": rpc error: code = NotFound desc = could not find container \"11654c6a755fd96a610bc8f62740d4275d0560427786092eadaa457d0a53e64b\": container with ID starting with 11654c6a755fd96a610bc8f62740d4275d0560427786092eadaa457d0a53e64b not found: ID does not exist"
Sep 14 03:07:40.089538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.089497    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"]
Sep 14 03:07:40.092816 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.092798    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4"]
Sep 14 03:07:40.105413 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.105398    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7edbe9e7-db64-421a-9330-98c14982d896-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:40.105472 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.105416    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:40.105472 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.105427    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7edbe9e7-db64-421a-9330-98c14982d896-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:40.105472 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.105436    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7edbe9e7-db64-421a-9330-98c14982d896-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:40.105472 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.105443    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:40.105472 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.105451    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gjjk\" (UniqueName: \"kubernetes.io/projected/7edbe9e7-db64-421a-9330-98c14982d896-kube-api-access-7gjjk\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:40.105472 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.105460    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:40.105472 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.105468    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:40.105696 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.105475    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7edbe9e7-db64-421a-9330-98c14982d896-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:40.215788 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.215744    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7edbe9e7-db64-421a-9330-98c14982d896" path="/var/lib/kubelet/pods/7edbe9e7-db64-421a-9330-98c14982d896/volumes"
Sep 14 03:07:40.737342 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:40.737299    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:07:45.825299 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:45.825266    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs"]
Sep 14 03:07:45.828165 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:45.828139    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bc1117-874fs"]
Sep 14 03:07:46.215963 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:46.215888    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14e73d3f-4423-4f61-9b31-0ef4787f929c" path="/var/lib/kubelet/pods/14e73d3f-4423-4f61-9b31-0ef4787f929c/volumes"
Sep 14 03:07:48.042557 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:48.042524    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:07:52.741134 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:52.741099    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"]
Sep 14 03:07:52.741569 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:52.741291    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f" podUID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerName="istio-proxy" containerID="cri-o://608bb2b1b030520dd7b5b992801084675c7aaae463ce919f612f1bc0e2b39f17" gracePeriod=30
Sep 14 03:07:56.763585 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:56.763550    2574 patch_prober.go:28] interesting pod/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 03:07:56.764013 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:56.763627    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f" podUID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 03:07:57.995390 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:57.995367    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:07:58.063986 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.063914    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-token\") pod \"426574c6-8729-47fd-bc40-2c612dfcd7ac\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") "
Sep 14 03:07:58.063986 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.063956    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-data\") pod \"426574c6-8729-47fd-bc40-2c612dfcd7ac\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") "
Sep 14 03:07:58.063986 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.063975    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-workload-certs\") pod \"426574c6-8729-47fd-bc40-2c612dfcd7ac\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") "
Sep 14 03:07:58.064243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.064014    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/426574c6-8729-47fd-bc40-2c612dfcd7ac-istiod-ca-cert\") pod \"426574c6-8729-47fd-bc40-2c612dfcd7ac\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") "
Sep 14 03:07:58.064243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.064045    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-credential-socket\") pod \"426574c6-8729-47fd-bc40-2c612dfcd7ac\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") "
Sep 14 03:07:58.064243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.064065    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-envoy\") pod \"426574c6-8729-47fd-bc40-2c612dfcd7ac\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") "
Sep 14 03:07:58.064243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.064090    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-workload-socket\") pod \"426574c6-8729-47fd-bc40-2c612dfcd7ac\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") "
Sep 14 03:07:58.064243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.064133    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-podinfo\") pod \"426574c6-8729-47fd-bc40-2c612dfcd7ac\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") "
Sep 14 03:07:58.064243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.064168    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgcb8\" (UniqueName: \"kubernetes.io/projected/426574c6-8729-47fd-bc40-2c612dfcd7ac-kube-api-access-xgcb8\") pod \"426574c6-8729-47fd-bc40-2c612dfcd7ac\" (UID: \"426574c6-8729-47fd-bc40-2c612dfcd7ac\") "
Sep 14 03:07:58.064243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.064201    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-data" (OuterVolumeSpecName: "istio-data") pod "426574c6-8729-47fd-bc40-2c612dfcd7ac" (UID: "426574c6-8729-47fd-bc40-2c612dfcd7ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:58.064604 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.064405    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:58.064604 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.064448    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "426574c6-8729-47fd-bc40-2c612dfcd7ac" (UID: "426574c6-8729-47fd-bc40-2c612dfcd7ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:58.064604 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.064458    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "426574c6-8729-47fd-bc40-2c612dfcd7ac" (UID: "426574c6-8729-47fd-bc40-2c612dfcd7ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:58.064604 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.064478    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/426574c6-8729-47fd-bc40-2c612dfcd7ac-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "426574c6-8729-47fd-bc40-2c612dfcd7ac" (UID: "426574c6-8729-47fd-bc40-2c612dfcd7ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:58.064781 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.064620    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "426574c6-8729-47fd-bc40-2c612dfcd7ac" (UID: "426574c6-8729-47fd-bc40-2c612dfcd7ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:58.066395 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.066366    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-token" (OuterVolumeSpecName: "istio-token") pod "426574c6-8729-47fd-bc40-2c612dfcd7ac" (UID: "426574c6-8729-47fd-bc40-2c612dfcd7ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:58.066615 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.066589    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/426574c6-8729-47fd-bc40-2c612dfcd7ac-kube-api-access-xgcb8" (OuterVolumeSpecName: "kube-api-access-xgcb8") pod "426574c6-8729-47fd-bc40-2c612dfcd7ac" (UID: "426574c6-8729-47fd-bc40-2c612dfcd7ac"). InnerVolumeSpecName "kube-api-access-xgcb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:58.066813 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.066797    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "426574c6-8729-47fd-bc40-2c612dfcd7ac" (UID: "426574c6-8729-47fd-bc40-2c612dfcd7ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:07:58.066876 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.066815    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "426574c6-8729-47fd-bc40-2c612dfcd7ac" (UID: "426574c6-8729-47fd-bc40-2c612dfcd7ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:58.157236 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.157201    2574 generic.go:356] "Generic (PLEG): container finished" podID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerID="608bb2b1b030520dd7b5b992801084675c7aaae463ce919f612f1bc0e2b39f17" exitCode=0
Sep 14 03:07:58.157433 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.157254    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"
Sep 14 03:07:58.157433 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.157279    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f" event={"ID":"426574c6-8729-47fd-bc40-2c612dfcd7ac","Type":"ContainerDied","Data":"608bb2b1b030520dd7b5b992801084675c7aaae463ce919f612f1bc0e2b39f17"}
Sep 14 03:07:58.157433 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.157338    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f" event={"ID":"426574c6-8729-47fd-bc40-2c612dfcd7ac","Type":"ContainerDied","Data":"e568591be4bb61b55c3a24f49d27225a588c3809fcfe203b84c1114f540317af"}
Sep 14 03:07:58.157433 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.157366    2574 scope.go:117] "RemoveContainer" containerID="608bb2b1b030520dd7b5b992801084675c7aaae463ce919f612f1bc0e2b39f17"
Sep 14 03:07:58.165145 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.165058    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:58.165145 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.165089    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:58.165145 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.165104    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgcb8\" (UniqueName: \"kubernetes.io/projected/426574c6-8729-47fd-bc40-2c612dfcd7ac-kube-api-access-xgcb8\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:58.165145 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.165133    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:58.165608 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.165146    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:58.165608 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.165173    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/426574c6-8729-47fd-bc40-2c612dfcd7ac-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:58.165608 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.165186    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:58.165608 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.165197    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/426574c6-8729-47fd-bc40-2c612dfcd7ac-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:07:58.167427 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.167406    2574 scope.go:117] "RemoveContainer" containerID="fb9826212932a644196f4902ac373b14af3e92745d63d5b907e8f7ffd80892fc"
Sep 14 03:07:58.175724 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.175705    2574 scope.go:117] "RemoveContainer" containerID="608bb2b1b030520dd7b5b992801084675c7aaae463ce919f612f1bc0e2b39f17"
Sep 14 03:07:58.176041 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:07:58.176023    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"608bb2b1b030520dd7b5b992801084675c7aaae463ce919f612f1bc0e2b39f17\": container with ID starting with 608bb2b1b030520dd7b5b992801084675c7aaae463ce919f612f1bc0e2b39f17 not found: ID does not exist" containerID="608bb2b1b030520dd7b5b992801084675c7aaae463ce919f612f1bc0e2b39f17"
Sep 14 03:07:58.176090 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.176051    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"608bb2b1b030520dd7b5b992801084675c7aaae463ce919f612f1bc0e2b39f17"} err="failed to get container status \"608bb2b1b030520dd7b5b992801084675c7aaae463ce919f612f1bc0e2b39f17\": rpc error: code = NotFound desc = could not find container \"608bb2b1b030520dd7b5b992801084675c7aaae463ce919f612f1bc0e2b39f17\": container with ID starting with 608bb2b1b030520dd7b5b992801084675c7aaae463ce919f612f1bc0e2b39f17 not found: ID does not exist"
Sep 14 03:07:58.176090 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.176070    2574 scope.go:117] "RemoveContainer" containerID="fb9826212932a644196f4902ac373b14af3e92745d63d5b907e8f7ffd80892fc"
Sep 14 03:07:58.176320 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:07:58.176302    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb9826212932a644196f4902ac373b14af3e92745d63d5b907e8f7ffd80892fc\": container with ID starting with fb9826212932a644196f4902ac373b14af3e92745d63d5b907e8f7ffd80892fc not found: ID does not exist" containerID="fb9826212932a644196f4902ac373b14af3e92745d63d5b907e8f7ffd80892fc"
Sep 14 03:07:58.176363 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.176327    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb9826212932a644196f4902ac373b14af3e92745d63d5b907e8f7ffd80892fc"} err="failed to get container status \"fb9826212932a644196f4902ac373b14af3e92745d63d5b907e8f7ffd80892fc\": rpc error: code = NotFound desc = could not find container \"fb9826212932a644196f4902ac373b14af3e92745d63d5b907e8f7ffd80892fc\": container with ID starting with fb9826212932a644196f4902ac373b14af3e92745d63d5b907e8f7ffd80892fc not found: ID does not exist"
Sep 14 03:07:58.180797 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.180777    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"]
Sep 14 03:07:58.190604 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.190581    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f"]
Sep 14 03:07:58.215868 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:07:58.215839    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="426574c6-8729-47fd-bc40-2c612dfcd7ac" path="/var/lib/kubelet/pods/426574c6-8729-47fd-bc40-2c612dfcd7ac/volumes"
Sep 14 03:08:01.338416 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:01.338373    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:08:22.374398 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.374354    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"]
Sep 14 03:08:22.374985 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.374855    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerName="istio-proxy"
Sep 14 03:08:22.374985 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.374875    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerName="istio-proxy"
Sep 14 03:08:22.374985 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.374889    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7edbe9e7-db64-421a-9330-98c14982d896" containerName="istio-proxy"
Sep 14 03:08:22.374985 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.374897    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7edbe9e7-db64-421a-9330-98c14982d896" containerName="istio-proxy"
Sep 14 03:08:22.374985 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.374909    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14e73d3f-4423-4f61-9b31-0ef4787f929c" containerName="revoke-keys"
Sep 14 03:08:22.374985 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.374917    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="14e73d3f-4423-4f61-9b31-0ef4787f929c" containerName="revoke-keys"
Sep 14 03:08:22.374985 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.374934    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerName="istio-proxy"
Sep 14 03:08:22.374985 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.374942    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerName="istio-proxy"
Sep 14 03:08:22.375461 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.375040    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7edbe9e7-db64-421a-9330-98c14982d896" containerName="istio-proxy"
Sep 14 03:08:22.375461 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.375051    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7edbe9e7-db64-421a-9330-98c14982d896" containerName="istio-proxy"
Sep 14 03:08:22.375461 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.375062    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerName="istio-proxy"
Sep 14 03:08:22.375461 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.375070    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="426574c6-8729-47fd-bc40-2c612dfcd7ac" containerName="istio-proxy"
Sep 14 03:08:22.375461 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.375079    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="14e73d3f-4423-4f61-9b31-0ef4787f929c" containerName="revoke-keys"
Sep 14 03:08:22.378090 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.378069    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.384453 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.384429    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-f83c87-openshift-default-dockercfg-x87xn\""
Sep 14 03:08:22.397128 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.397071    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"]
Sep 14 03:08:22.460454 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.460413    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa86f69b-8639-4711-a69b-4872e6f56368-istiod-ca-cert\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.460454 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.460454    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-workload-certs\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.460710 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.460471    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa86f69b-8639-4711-a69b-4872e6f56368-istio-token\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.460710 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.460566    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-istio-envoy\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.460710 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.460613    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-istio-data\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.460710 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.460652    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr6nl\" (UniqueName: \"kubernetes.io/projected/aa86f69b-8639-4711-a69b-4872e6f56368-kube-api-access-dr6nl\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.460710 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.460692    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-credential-socket\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.460710 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.460707    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa86f69b-8639-4711-a69b-4872e6f56368-istio-podinfo\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.460971 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.460793    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-workload-socket\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.562010 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.561967    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-credential-socket\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.562010 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.562013    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa86f69b-8639-4711-a69b-4872e6f56368-istio-podinfo\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.562272 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.562144    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-workload-socket\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.562272 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.562212    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa86f69b-8639-4711-a69b-4872e6f56368-istiod-ca-cert\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.562272 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.562232    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-workload-certs\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.562272 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.562250    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa86f69b-8639-4711-a69b-4872e6f56368-istio-token\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.562483 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.562284    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-istio-envoy\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.562483 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.562302    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-istio-data\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.562483 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.562465    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-credential-socket\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.562695 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.562483    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-workload-socket\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.562695 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.562554    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr6nl\" (UniqueName: \"kubernetes.io/projected/aa86f69b-8639-4711-a69b-4872e6f56368-kube-api-access-dr6nl\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.562894 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.562845    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-workload-certs\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.562975 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.562912    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-istio-data\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.563198 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.563122    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa86f69b-8639-4711-a69b-4872e6f56368-istiod-ca-cert\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.564685 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.564663    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-istio-envoy\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.564779 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.564768    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa86f69b-8639-4711-a69b-4872e6f56368-istio-podinfo\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.573088 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.573061    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa86f69b-8639-4711-a69b-4872e6f56368-istio-token\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.573928 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.573912    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr6nl\" (UniqueName: \"kubernetes.io/projected/aa86f69b-8639-4711-a69b-4872e6f56368-kube-api-access-dr6nl\") pod \"e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") " pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.688624 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.688531    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:22.973434 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.973411    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"]
Sep 14 03:08:22.976100 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:08:22.976066    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa86f69b_8639_4711_a69b_4872e6f56368.slice/crio-7f6321a4f74d5a0417ad4521aa01c1a19e8cad372832a261fdb467a19f1c1fc3 WatchSource:0}: Error finding container 7f6321a4f74d5a0417ad4521aa01c1a19e8cad372832a261fdb467a19f1c1fc3: Status 404 returned error can't find the container with id 7f6321a4f74d5a0417ad4521aa01c1a19e8cad372832a261fdb467a19f1c1fc3
Sep 14 03:08:22.980774 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.980739    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:08:22.980893 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.980817    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:08:22.980893 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:22.980866    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:08:23.282072 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:23.282037    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9" event={"ID":"aa86f69b-8639-4711-a69b-4872e6f56368","Type":"ContainerStarted","Data":"f098b9b98f48b43716fc8847239330e587197e338e494791764233a479641a83"}
Sep 14 03:08:23.282255 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:23.282091    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9" event={"ID":"aa86f69b-8639-4711-a69b-4872e6f56368","Type":"ContainerStarted","Data":"7f6321a4f74d5a0417ad4521aa01c1a19e8cad372832a261fdb467a19f1c1fc3"}
Sep 14 03:08:23.305850 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:23.305790    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9" podStartSLOduration=1.305768682 podStartE2EDuration="1.305768682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:08:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:08:23.30385698 +0000 UTC m=+1049.616880800" watchObservedRunningTime="2026-09-14 03:08:23.305768682 +0000 UTC m=+1049.618792502"
Sep 14 03:08:23.688670 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:23.688587    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:24.693898 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:24.693872    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:25.292207 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:25.292177    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:25.293134 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:25.293114    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:31.639352 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:31.639313    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:08:41.110708 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:41.109864    2574 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:56104->10.134.0.41:15021: read: connection reset by peer" start-of-body=
Sep 14 03:08:41.110708 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:41.109950    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9" podUID="aa86f69b-8639-4711-a69b-4872e6f56368" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:56104->10.134.0.41:15021: read: connection reset by peer"
Sep 14 03:08:41.379281 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:41.379185    2574 generic.go:356] "Generic (PLEG): container finished" podID="aa86f69b-8639-4711-a69b-4872e6f56368" containerID="f098b9b98f48b43716fc8847239330e587197e338e494791764233a479641a83" exitCode=0
Sep 14 03:08:41.379281 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:41.379252    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9" event={"ID":"aa86f69b-8639-4711-a69b-4872e6f56368","Type":"ContainerDied","Data":"f098b9b98f48b43716fc8847239330e587197e338e494791764233a479641a83"}
Sep 14 03:08:41.379682 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:41.379668    2574 scope.go:117] "RemoveContainer" containerID="f098b9b98f48b43716fc8847239330e587197e338e494791764233a479641a83"
Sep 14 03:08:41.380722 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:41.380695    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:08:41.380796 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:41.380765    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:08:41.380833 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:41.380795    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:08:42.389814 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:42.389773    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9" event={"ID":"aa86f69b-8639-4711-a69b-4872e6f56368","Type":"ContainerStarted","Data":"ace602a9e6cf8f7d40899ec64bb26fbe9979a8a34ac379edfa0001ec0e0278cc"}
Sep 14 03:08:42.689687 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:42.689575    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:42.690014 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:42.689710    2574 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 14 03:08:42.690203 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:42.690169    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9" podUID="aa86f69b-8639-4711-a69b-4872e6f56368" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 14 03:08:43.690019 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:43.689964    2574 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 14 03:08:43.690568 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:43.690039    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9" podUID="aa86f69b-8639-4711-a69b-4872e6f56368" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 14 03:08:44.689426 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:44.689374    2574 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 14 03:08:44.689667 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:44.689449    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9" podUID="aa86f69b-8639-4711-a69b-4872e6f56368" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 14 03:08:45.690031 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:45.689917    2574 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 14 03:08:45.690031 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:45.689993    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9" podUID="aa86f69b-8639-4711-a69b-4872e6f56368" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 14 03:08:46.689273 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:46.689235    2574 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 14 03:08:46.689598 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:46.689567    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9" podUID="aa86f69b-8639-4711-a69b-4872e6f56368" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 14 03:08:47.692926 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:47.692890    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:47.693395 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:47.693276    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:47.693998 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:47.693979    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:08:57.985163 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:57.985134    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"]
Sep 14 03:08:57.985563 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:57.985479    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7edbe9e7-db64-421a-9330-98c14982d896" containerName="istio-proxy"
Sep 14 03:08:57.985563 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:57.985491    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7edbe9e7-db64-421a-9330-98c14982d896" containerName="istio-proxy"
Sep 14 03:08:57.989433 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:57.989418    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:57.993362 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:57.993333    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-0c919e\"/\"auth-test-model-0c919e-kserve-self-signed-certs\""
Sep 14 03:08:57.993362 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:57.993345    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-0c919e\"/\"kube-root-ca.crt\""
Sep 14 03:08:57.993362 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:57.993356    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-0c919e\"/\"default-dockercfg-llp22\""
Sep 14 03:08:57.993609 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:57.993341    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-0c919e\"/\"openshift-service-ca.crt\""
Sep 14 03:08:57.999571 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:57.999549    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"]
Sep 14 03:08:58.077387 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.077076    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-dshm\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.077387 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.077144    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-home\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.077387 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.077194    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6zqh\" (UniqueName: \"kubernetes.io/projected/9b6ef270-f2f5-48ac-8b8a-8e4297318330-kube-api-access-g6zqh\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.077387 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.077234    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-tmp-dir\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.077387 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.077256    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b6ef270-f2f5-48ac-8b8a-8e4297318330-tls-certs\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.077387 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.077305    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-model-cache\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.178350 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.178323    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-home\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.178481 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.178365    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6zqh\" (UniqueName: \"kubernetes.io/projected/9b6ef270-f2f5-48ac-8b8a-8e4297318330-kube-api-access-g6zqh\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.178481 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.178394    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-tmp-dir\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.178481 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.178408    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b6ef270-f2f5-48ac-8b8a-8e4297318330-tls-certs\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.178481 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.178448    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-model-cache\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.178481 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.178471    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-dshm\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.178823 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.178797    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-tmp-dir\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.178886 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.178830    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-home\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.178925 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.178902    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-model-cache\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.180917 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.180890    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-dshm\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.181064 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.181049    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b6ef270-f2f5-48ac-8b8a-8e4297318330-tls-certs\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.188433 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.188411    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6zqh\" (UniqueName: \"kubernetes.io/projected/9b6ef270-f2f5-48ac-8b8a-8e4297318330-kube-api-access-g6zqh\") pod \"auth-test-model-0c919e-kserve-75bdf8695f-7pkdp\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.299594 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.299496    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:08:58.570783 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:58.570743    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"]
Sep 14 03:08:58.571766 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:08:58.571743    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b6ef270_f2f5_48ac_8b8a_8e4297318330.slice/crio-bf4be3807670f2a21a4075cd195f405c7e3886b820c5adc903e5813da2937381 WatchSource:0}: Error finding container bf4be3807670f2a21a4075cd195f405c7e3886b820c5adc903e5813da2937381: Status 404 returned error can't find the container with id bf4be3807670f2a21a4075cd195f405c7e3886b820c5adc903e5813da2937381
Sep 14 03:08:59.035989 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:59.035959    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:08:59.473195 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:59.473111    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp" event={"ID":"9b6ef270-f2f5-48ac-8b8a-8e4297318330","Type":"ContainerStarted","Data":"5b33497d5f5d58c81158ca8c6cc50572cb5909b70328502e5de24191175db777"}
Sep 14 03:08:59.473195 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:59.473154    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp" event={"ID":"9b6ef270-f2f5-48ac-8b8a-8e4297318330","Type":"ContainerStarted","Data":"bf4be3807670f2a21a4075cd195f405c7e3886b820c5adc903e5813da2937381"}
Sep 14 03:08:59.493236 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:08:59.493195    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp" podStartSLOduration=2.493177015 podStartE2EDuration="2.493177015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:08:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:08:59.491640279 +0000 UTC m=+1085.804664098" watchObservedRunningTime="2026-09-14 03:08:59.493177015 +0000 UTC m=+1085.806200837"
Sep 14 03:09:08.299781 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:08.299753    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:09:08.300153 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:08.299793    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:09:08.312248 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:08.312222    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:09:08.527537 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:08.527486    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:09:09.336988 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:09.336947    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:09:11.690601 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.690570    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"]
Sep 14 03:09:11.694305 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.694280    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.697940 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.697920    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-dfbaeab8-openshift-default-dockercfg-gv5l5\""
Sep 14 03:09:11.708635 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.708613    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"]
Sep 14 03:09:11.790660 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.790626    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-credential-socket\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.790660 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.790668    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-workload-socket\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.790893 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.790694    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw7hg\" (UniqueName: \"kubernetes.io/projected/b9b31752-5e13-47c7-992a-e79834207f29-kube-api-access-bw7hg\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.790893 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.790752    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9b31752-5e13-47c7-992a-e79834207f29-istio-token\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.790893 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.790822    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9b31752-5e13-47c7-992a-e79834207f29-istiod-ca-cert\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.791058 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.790918    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-istio-data\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.791058 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.790966    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-istio-envoy\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.791058 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.791000    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-workload-certs\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.791058 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.791037    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9b31752-5e13-47c7-992a-e79834207f29-istio-podinfo\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.891977 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.891930    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-istio-data\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.892170 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.891992    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-istio-envoy\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.892170 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.892015    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-workload-certs\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.892170 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.892039    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9b31752-5e13-47c7-992a-e79834207f29-istio-podinfo\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.892170 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.892079    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-credential-socket\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.892170 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.892098    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-workload-socket\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.892170 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.892113    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw7hg\" (UniqueName: \"kubernetes.io/projected/b9b31752-5e13-47c7-992a-e79834207f29-kube-api-access-bw7hg\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.892500 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.892261    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9b31752-5e13-47c7-992a-e79834207f29-istio-token\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.892500 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.892295    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9b31752-5e13-47c7-992a-e79834207f29-istiod-ca-cert\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.892500 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.892403    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-workload-certs\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.892500 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.892434    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-istio-data\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.892933 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.892912    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-workload-socket\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.893032 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.893018    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9b31752-5e13-47c7-992a-e79834207f29-istiod-ca-cert\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.893085 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.893062    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-credential-socket\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.894562 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.894547    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-istio-envoy\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.894810 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.894794    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9b31752-5e13-47c7-992a-e79834207f29-istio-podinfo\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.903804 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.903784    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9b31752-5e13-47c7-992a-e79834207f29-istio-token\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:11.907200 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:11.907171    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw7hg\" (UniqueName: \"kubernetes.io/projected/b9b31752-5e13-47c7-992a-e79834207f29-kube-api-access-bw7hg\") pod \"e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") " pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:12.005470 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:12.005433    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:12.285332 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:12.285306    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"]
Sep 14 03:09:12.287732 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:09:12.287702    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9b31752_5e13_47c7_992a_e79834207f29.slice/crio-393e86f0bc0660d5c667b23b238f78d1bde8472ca0f61c269daeed0e457b6351 WatchSource:0}: Error finding container 393e86f0bc0660d5c667b23b238f78d1bde8472ca0f61c269daeed0e457b6351: Status 404 returned error can't find the container with id 393e86f0bc0660d5c667b23b238f78d1bde8472ca0f61c269daeed0e457b6351
Sep 14 03:09:12.289906 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:12.289877    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:09:12.289966 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:12.289941    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:09:12.290015 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:12.289969    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:09:12.537401 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:12.537308    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd" event={"ID":"b9b31752-5e13-47c7-992a-e79834207f29","Type":"ContainerStarted","Data":"f8ca8333bd4a18247c679c7d6bf34aeb4fb6dba5251e566722f05cfe40fde6d0"}
Sep 14 03:09:12.537401 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:12.537361    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd" event={"ID":"b9b31752-5e13-47c7-992a-e79834207f29","Type":"ContainerStarted","Data":"393e86f0bc0660d5c667b23b238f78d1bde8472ca0f61c269daeed0e457b6351"}
Sep 14 03:09:12.560160 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:12.560109    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd" podStartSLOduration=1.5600927599999999 podStartE2EDuration="1.56009276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:09:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:09:12.559337538 +0000 UTC m=+1098.872361356" watchObservedRunningTime="2026-09-14 03:09:12.56009276 +0000 UTC m=+1098.873116578"
Sep 14 03:09:13.006586 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:13.006561    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:13.011359 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:13.011334    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:13.542197 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:13.542163    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:13.542997 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:13.542979    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:21.749793 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:21.749760    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:09:32.642346 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:32.642313    2574 generic.go:356] "Generic (PLEG): container finished" podID="b9b31752-5e13-47c7-992a-e79834207f29" containerID="f8ca8333bd4a18247c679c7d6bf34aeb4fb6dba5251e566722f05cfe40fde6d0" exitCode=0
Sep 14 03:09:32.642788 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:32.642366    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd" event={"ID":"b9b31752-5e13-47c7-992a-e79834207f29","Type":"ContainerDied","Data":"f8ca8333bd4a18247c679c7d6bf34aeb4fb6dba5251e566722f05cfe40fde6d0"}
Sep 14 03:09:32.642833 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:32.642785    2574 scope.go:117] "RemoveContainer" containerID="f8ca8333bd4a18247c679c7d6bf34aeb4fb6dba5251e566722f05cfe40fde6d0"
Sep 14 03:09:32.643817 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:32.643789    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:09:32.643881 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:32.643856    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:09:32.643917 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:32.643887    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:09:33.649605 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:33.649571    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd" event={"ID":"b9b31752-5e13-47c7-992a-e79834207f29","Type":"ContainerStarted","Data":"c32aaeb140b3d0efa19d42ac353799c5b6a2f0861a2c024b3b3b5653dad41011"}
Sep 14 03:09:34.006088 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:34.006052    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:34.006311 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:34.006289    2574 patch_prober.go:28] interesting pod/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 14 03:09:34.006358 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:34.006340    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd" podUID="b9b31752-5e13-47c7-992a-e79834207f29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 14 03:09:35.006195 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:35.006157    2574 patch_prober.go:28] interesting pod/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 14 03:09:35.006644 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:35.006226    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd" podUID="b9b31752-5e13-47c7-992a-e79834207f29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 14 03:09:36.006591 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:36.006555    2574 patch_prober.go:28] interesting pod/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 14 03:09:36.007004 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:36.006615    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd" podUID="b9b31752-5e13-47c7-992a-e79834207f29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 14 03:09:37.006909 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:37.006873    2574 patch_prober.go:28] interesting pod/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 14 03:09:37.007295 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:37.006947    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd" podUID="b9b31752-5e13-47c7-992a-e79834207f29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 14 03:09:38.006773 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:38.006735    2574 patch_prober.go:28] interesting pod/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 14 03:09:38.006948 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:38.006807    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd" podUID="b9b31752-5e13-47c7-992a-e79834207f29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 14 03:09:39.010502 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:39.010466    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:39.010976 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:39.010749    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:39.011417 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:39.011390    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:09:42.085353 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.085299    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"]
Sep 14 03:09:42.088909 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.088891    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.091797 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.091772    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-f83c87\"/\"openshift-service-ca.crt\""
Sep 14 03:09:42.091905 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.091784    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-f83c87\"/\"kube-root-ca.crt\""
Sep 14 03:09:42.092799 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.092777    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-f83c87\"/\"auth-test-model-f83c87-kserve-self-signed-certs\""
Sep 14 03:09:42.092799 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.092792    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-f83c87\"/\"default-dockercfg-j84w5\""
Sep 14 03:09:42.100431 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.100408    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"]
Sep 14 03:09:42.153763 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.153738    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-dshm\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.153864 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.153765    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080026c0-93d7-4fc0-8807-999827fcd7c7-tls-certs\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.153864 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.153796    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-model-cache\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.153864 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.153843    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-home\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.153981 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.153893    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7p4l\" (UniqueName: \"kubernetes.io/projected/080026c0-93d7-4fc0-8807-999827fcd7c7-kube-api-access-r7p4l\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.153981 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.153931    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-tmp-dir\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.254392 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.254352    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-dshm\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.254392 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.254392    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080026c0-93d7-4fc0-8807-999827fcd7c7-tls-certs\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.254626 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.254544    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-model-cache\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.254626 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.254576    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-home\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.254626 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.254620    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7p4l\" (UniqueName: \"kubernetes.io/projected/080026c0-93d7-4fc0-8807-999827fcd7c7-kube-api-access-r7p4l\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.254773 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.254658    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-tmp-dir\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.254918 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.254902    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-home\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.255014 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.254904    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-model-cache\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.255014 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.254975    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-tmp-dir\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.256825 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.256805    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-dshm\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.257198 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.257179    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080026c0-93d7-4fc0-8807-999827fcd7c7-tls-certs\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.262854 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.262833    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7p4l\" (UniqueName: \"kubernetes.io/projected/080026c0-93d7-4fc0-8807-999827fcd7c7-kube-api-access-r7p4l\") pod \"auth-test-model-f83c87-kserve-69758b87dc-qblsb\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.399382 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.399299    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:42.672556 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.672527    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"]
Sep 14 03:09:42.673334 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:09:42.673307    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod080026c0_93d7_4fc0_8807_999827fcd7c7.slice/crio-fb87ddcf933a21d0773795fb018dc8bab966fac71003c1d1b59f787664b6eb0f WatchSource:0}: Error finding container fb87ddcf933a21d0773795fb018dc8bab966fac71003c1d1b59f787664b6eb0f: Status 404 returned error can't find the container with id fb87ddcf933a21d0773795fb018dc8bab966fac71003c1d1b59f787664b6eb0f
Sep 14 03:09:42.696333 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:42.696309    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb" event={"ID":"080026c0-93d7-4fc0-8807-999827fcd7c7","Type":"ContainerStarted","Data":"fb87ddcf933a21d0773795fb018dc8bab966fac71003c1d1b59f787664b6eb0f"}
Sep 14 03:09:43.704302 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:43.704271    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb" event={"ID":"080026c0-93d7-4fc0-8807-999827fcd7c7","Type":"ContainerStarted","Data":"5df9ed368f22ef41f6edd9010063031e9f679d75003d8bf95b0cbef8ddb835a1"}
Sep 14 03:09:43.722092 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:43.722038    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb" podStartSLOduration=1.7219879790000001 podStartE2EDuration="1.721987979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:09:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:09:43.720838761 +0000 UTC m=+1130.033862607" watchObservedRunningTime="2026-09-14 03:09:43.721987979 +0000 UTC m=+1130.035011802"
Sep 14 03:09:46.531439 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:46.531401    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:09:47.293074 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:47.293044    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb"]
Sep 14 03:09:47.296632 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:47.296616    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb"
Sep 14 03:09:47.299466 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:47.299445    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4hf5p\""
Sep 14 03:09:47.299584 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:47.299533    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:09:47.307900 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:47.307879    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb"]
Sep 14 03:09:47.397648 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:47.397616    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f625d2-0f43-4cee-8bcb-528dfa013740-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb\" (UID: \"c1f625d2-0f43-4cee-8bcb-528dfa013740\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb"
Sep 14 03:09:47.498482 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:47.498451    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f625d2-0f43-4cee-8bcb-528dfa013740-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb\" (UID: \"c1f625d2-0f43-4cee-8bcb-528dfa013740\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb"
Sep 14 03:09:47.499080 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:47.499061    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f625d2-0f43-4cee-8bcb-528dfa013740-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb\" (UID: \"c1f625d2-0f43-4cee-8bcb-528dfa013740\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb"
Sep 14 03:09:47.609561 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:47.609492    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb"
Sep 14 03:09:47.878474 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:47.878450    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb"]
Sep 14 03:09:47.880069 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:09:47.880038    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f625d2_0f43_4cee_8bcb_528dfa013740.slice/crio-220a4ba9f295de9263531ff14f62d772e78c357832434f0cd50b8aa6222b08f0 WatchSource:0}: Error finding container 220a4ba9f295de9263531ff14f62d772e78c357832434f0cd50b8aa6222b08f0: Status 404 returned error can't find the container with id 220a4ba9f295de9263531ff14f62d772e78c357832434f0cd50b8aa6222b08f0
Sep 14 03:09:48.731429 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:48.731400    2574 generic.go:356] "Generic (PLEG): container finished" podID="c1f625d2-0f43-4cee-8bcb-528dfa013740" containerID="fab86dc33f54006c453e046ee67204def360c0561a27c3d663b1df15a11c7da9" exitCode=0
Sep 14 03:09:48.731770 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:48.731453    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb" event={"ID":"c1f625d2-0f43-4cee-8bcb-528dfa013740","Type":"ContainerDied","Data":"fab86dc33f54006c453e046ee67204def360c0561a27c3d663b1df15a11c7da9"}
Sep 14 03:09:48.731770 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:48.731526    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb" event={"ID":"c1f625d2-0f43-4cee-8bcb-528dfa013740","Type":"ContainerStarted","Data":"220a4ba9f295de9263531ff14f62d772e78c357832434f0cd50b8aa6222b08f0"}
Sep 14 03:09:49.871411 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:49.871383    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb"
Sep 14 03:09:49.921095 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:49.921067    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f625d2-0f43-4cee-8bcb-528dfa013740-maas-api-service-ca\") pod \"c1f625d2-0f43-4cee-8bcb-528dfa013740\" (UID: \"c1f625d2-0f43-4cee-8bcb-528dfa013740\") "
Sep 14 03:09:49.921430 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:49.921404    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1f625d2-0f43-4cee-8bcb-528dfa013740-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c1f625d2-0f43-4cee-8bcb-528dfa013740" (UID: "c1f625d2-0f43-4cee-8bcb-528dfa013740"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:09:50.022703 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:50.022619    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f625d2-0f43-4cee-8bcb-528dfa013740-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:09:50.743209 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:50.743172    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb" event={"ID":"c1f625d2-0f43-4cee-8bcb-528dfa013740","Type":"ContainerDied","Data":"220a4ba9f295de9263531ff14f62d772e78c357832434f0cd50b8aa6222b08f0"}
Sep 14 03:09:50.743209 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:50.743204    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb"
Sep 14 03:09:50.743427 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:50.743219    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="220a4ba9f295de9263531ff14f62d772e78c357832434f0cd50b8aa6222b08f0"
Sep 14 03:09:52.399700 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:52.399662    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:52.400093 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:52.399718    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:52.413300 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:52.413269    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:52.764771 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:52.764745    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:09:58.133943 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:58.133908    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb"]
Sep 14 03:09:58.137715 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:58.137690    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfbaeab8-rk6mb"]
Sep 14 03:09:58.216287 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:09:58.216261    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1f625d2-0f43-4cee-8bcb-528dfa013740" path="/var/lib/kubelet/pods/c1f625d2-0f43-4cee-8bcb-528dfa013740/volumes"
Sep 14 03:10:02.445403 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:02.445371    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:10:05.315457 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:05.315422    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"]
Sep 14 03:10:05.315861 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:05.315678    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd" podUID="b9b31752-5e13-47c7-992a-e79834207f29" containerName="istio-proxy" containerID="cri-o://c32aaeb140b3d0efa19d42ac353799c5b6a2f0861a2c024b3b3b5653dad41011" gracePeriod=30
Sep 14 03:10:09.013657 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:09.013614    2574 patch_prober.go:28] interesting pod/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 14 03:10:09.014056 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:09.013697    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd" podUID="b9b31752-5e13-47c7-992a-e79834207f29" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 03:10:10.570907 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.570885    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:10:10.714856 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.714753    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-istio-envoy\") pod \"b9b31752-5e13-47c7-992a-e79834207f29\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") "
Sep 14 03:10:10.714856 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.714803    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-workload-certs\") pod \"b9b31752-5e13-47c7-992a-e79834207f29\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") "
Sep 14 03:10:10.714856 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.714830    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9b31752-5e13-47c7-992a-e79834207f29-istio-podinfo\") pod \"b9b31752-5e13-47c7-992a-e79834207f29\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") "
Sep 14 03:10:10.715119 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.714862    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9b31752-5e13-47c7-992a-e79834207f29-istio-token\") pod \"b9b31752-5e13-47c7-992a-e79834207f29\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") "
Sep 14 03:10:10.715119 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.714901    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bw7hg\" (UniqueName: \"kubernetes.io/projected/b9b31752-5e13-47c7-992a-e79834207f29-kube-api-access-bw7hg\") pod \"b9b31752-5e13-47c7-992a-e79834207f29\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") "
Sep 14 03:10:10.715119 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.714955    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-credential-socket\") pod \"b9b31752-5e13-47c7-992a-e79834207f29\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") "
Sep 14 03:10:10.715119 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.714982    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9b31752-5e13-47c7-992a-e79834207f29-istiod-ca-cert\") pod \"b9b31752-5e13-47c7-992a-e79834207f29\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") "
Sep 14 03:10:10.715119 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.715074    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-workload-socket\") pod \"b9b31752-5e13-47c7-992a-e79834207f29\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") "
Sep 14 03:10:10.715119 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.715071    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b9b31752-5e13-47c7-992a-e79834207f29" (UID: "b9b31752-5e13-47c7-992a-e79834207f29"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:10:10.715119 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.715103    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-istio-data\") pod \"b9b31752-5e13-47c7-992a-e79834207f29\" (UID: \"b9b31752-5e13-47c7-992a-e79834207f29\") "
Sep 14 03:10:10.715444 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.715394    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b9b31752-5e13-47c7-992a-e79834207f29" (UID: "b9b31752-5e13-47c7-992a-e79834207f29"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:10:10.715505 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.715406    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:10:10.715505 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.715448    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b9b31752-5e13-47c7-992a-e79834207f29" (UID: "b9b31752-5e13-47c7-992a-e79834207f29"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:10:10.715505 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.715454    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9b31752-5e13-47c7-992a-e79834207f29-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b9b31752-5e13-47c7-992a-e79834207f29" (UID: "b9b31752-5e13-47c7-992a-e79834207f29"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:10:10.715691 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.715604    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-istio-data" (OuterVolumeSpecName: "istio-data") pod "b9b31752-5e13-47c7-992a-e79834207f29" (UID: "b9b31752-5e13-47c7-992a-e79834207f29"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:10:10.717409 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.717373    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9b31752-5e13-47c7-992a-e79834207f29-kube-api-access-bw7hg" (OuterVolumeSpecName: "kube-api-access-bw7hg") pod "b9b31752-5e13-47c7-992a-e79834207f29" (UID: "b9b31752-5e13-47c7-992a-e79834207f29"). InnerVolumeSpecName "kube-api-access-bw7hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:10:10.717547 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.717480    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9b31752-5e13-47c7-992a-e79834207f29-istio-token" (OuterVolumeSpecName: "istio-token") pod "b9b31752-5e13-47c7-992a-e79834207f29" (UID: "b9b31752-5e13-47c7-992a-e79834207f29"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:10:10.717825 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.717805    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b9b31752-5e13-47c7-992a-e79834207f29-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b9b31752-5e13-47c7-992a-e79834207f29" (UID: "b9b31752-5e13-47c7-992a-e79834207f29"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:10:10.717963 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.717850    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b9b31752-5e13-47c7-992a-e79834207f29" (UID: "b9b31752-5e13-47c7-992a-e79834207f29"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:10:10.816140 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.816116    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:10:10.816140 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.816140    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9b31752-5e13-47c7-992a-e79834207f29-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:10:10.816257 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.816149    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:10:10.816257 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.816158    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:10:10.816257 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.816168    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9b31752-5e13-47c7-992a-e79834207f29-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:10:10.816257 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.816176    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9b31752-5e13-47c7-992a-e79834207f29-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:10:10.816257 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.816183    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9b31752-5e13-47c7-992a-e79834207f29-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:10:10.816257 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.816190    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw7hg\" (UniqueName: \"kubernetes.io/projected/b9b31752-5e13-47c7-992a-e79834207f29-kube-api-access-bw7hg\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:10:10.851507 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.851481    2574 generic.go:356] "Generic (PLEG): container finished" podID="b9b31752-5e13-47c7-992a-e79834207f29" containerID="c32aaeb140b3d0efa19d42ac353799c5b6a2f0861a2c024b3b3b5653dad41011" exitCode=0
Sep 14 03:10:10.851631 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.851545    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd" event={"ID":"b9b31752-5e13-47c7-992a-e79834207f29","Type":"ContainerDied","Data":"c32aaeb140b3d0efa19d42ac353799c5b6a2f0861a2c024b3b3b5653dad41011"}
Sep 14 03:10:10.851631 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.851557    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"
Sep 14 03:10:10.851631 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.851595    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd" event={"ID":"b9b31752-5e13-47c7-992a-e79834207f29","Type":"ContainerDied","Data":"393e86f0bc0660d5c667b23b238f78d1bde8472ca0f61c269daeed0e457b6351"}
Sep 14 03:10:10.851631 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.851617    2574 scope.go:117] "RemoveContainer" containerID="c32aaeb140b3d0efa19d42ac353799c5b6a2f0861a2c024b3b3b5653dad41011"
Sep 14 03:10:10.860704 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.860687    2574 scope.go:117] "RemoveContainer" containerID="f8ca8333bd4a18247c679c7d6bf34aeb4fb6dba5251e566722f05cfe40fde6d0"
Sep 14 03:10:10.868971 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.868955    2574 scope.go:117] "RemoveContainer" containerID="c32aaeb140b3d0efa19d42ac353799c5b6a2f0861a2c024b3b3b5653dad41011"
Sep 14 03:10:10.869237 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:10:10.869220    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c32aaeb140b3d0efa19d42ac353799c5b6a2f0861a2c024b3b3b5653dad41011\": container with ID starting with c32aaeb140b3d0efa19d42ac353799c5b6a2f0861a2c024b3b3b5653dad41011 not found: ID does not exist" containerID="c32aaeb140b3d0efa19d42ac353799c5b6a2f0861a2c024b3b3b5653dad41011"
Sep 14 03:10:10.869279 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.869245    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c32aaeb140b3d0efa19d42ac353799c5b6a2f0861a2c024b3b3b5653dad41011"} err="failed to get container status \"c32aaeb140b3d0efa19d42ac353799c5b6a2f0861a2c024b3b3b5653dad41011\": rpc error: code = NotFound desc = could not find container \"c32aaeb140b3d0efa19d42ac353799c5b6a2f0861a2c024b3b3b5653dad41011\": container with ID starting with c32aaeb140b3d0efa19d42ac353799c5b6a2f0861a2c024b3b3b5653dad41011 not found: ID does not exist"
Sep 14 03:10:10.869279 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.869262    2574 scope.go:117] "RemoveContainer" containerID="f8ca8333bd4a18247c679c7d6bf34aeb4fb6dba5251e566722f05cfe40fde6d0"
Sep 14 03:10:10.869472 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:10:10.869458    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8ca8333bd4a18247c679c7d6bf34aeb4fb6dba5251e566722f05cfe40fde6d0\": container with ID starting with f8ca8333bd4a18247c679c7d6bf34aeb4fb6dba5251e566722f05cfe40fde6d0 not found: ID does not exist" containerID="f8ca8333bd4a18247c679c7d6bf34aeb4fb6dba5251e566722f05cfe40fde6d0"
Sep 14 03:10:10.869528 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.869476    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8ca8333bd4a18247c679c7d6bf34aeb4fb6dba5251e566722f05cfe40fde6d0"} err="failed to get container status \"f8ca8333bd4a18247c679c7d6bf34aeb4fb6dba5251e566722f05cfe40fde6d0\": rpc error: code = NotFound desc = could not find container \"f8ca8333bd4a18247c679c7d6bf34aeb4fb6dba5251e566722f05cfe40fde6d0\": container with ID starting with f8ca8333bd4a18247c679c7d6bf34aeb4fb6dba5251e566722f05cfe40fde6d0 not found: ID does not exist"
Sep 14 03:10:10.874746 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.874725    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"]
Sep 14 03:10:10.880396 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:10.880376    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd"]
Sep 14 03:10:12.215403 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:12.215370    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9b31752-5e13-47c7-992a-e79834207f29" path="/var/lib/kubelet/pods/b9b31752-5e13-47c7-992a-e79834207f29/volumes"
Sep 14 03:10:15.736975 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:15.736892    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:10:20.336222 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.336187    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"]
Sep 14 03:10:20.336615 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.336564    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9b31752-5e13-47c7-992a-e79834207f29" containerName="istio-proxy"
Sep 14 03:10:20.336615 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.336576    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9b31752-5e13-47c7-992a-e79834207f29" containerName="istio-proxy"
Sep 14 03:10:20.336615 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.336587    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f625d2-0f43-4cee-8bcb-528dfa013740" containerName="revoke-keys"
Sep 14 03:10:20.336615 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.336593    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f625d2-0f43-4cee-8bcb-528dfa013740" containerName="revoke-keys"
Sep 14 03:10:20.336615 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.336608    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9b31752-5e13-47c7-992a-e79834207f29" containerName="istio-proxy"
Sep 14 03:10:20.336615 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.336615    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9b31752-5e13-47c7-992a-e79834207f29" containerName="istio-proxy"
Sep 14 03:10:20.336824 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.336684    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9b31752-5e13-47c7-992a-e79834207f29" containerName="istio-proxy"
Sep 14 03:10:20.336824 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.336696    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f625d2-0f43-4cee-8bcb-528dfa013740" containerName="revoke-keys"
Sep 14 03:10:20.341153 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.341134    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.343907 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.343888    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-6e87d0-openshift-default-dockercfg-qzm5n\""
Sep 14 03:10:20.350902 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.350878    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"]
Sep 14 03:10:20.393212 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.393175    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhxh8\" (UniqueName: \"kubernetes.io/projected/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-kube-api-access-dhxh8\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.393385 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.393231    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-workload-certs\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.393385 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.393304    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-envoy\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.393385 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.393342    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-data\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.393385 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.393362    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-token\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.393385 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.393378    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-workload-socket\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.393585 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.393438    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istiod-ca-cert\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.393585 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.393471    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-podinfo\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.393585 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.393507    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-credential-socket\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.494737 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.494702    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-workload-certs\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.494945 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.494750    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-envoy\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.494945 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.494793    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-data\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.494945 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.494819    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-token\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.494945 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.494848    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-workload-socket\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.495166 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.495003    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istiod-ca-cert\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.495166 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.495038    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-podinfo\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.495166 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.495094    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-credential-socket\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.495301 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.495170    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhxh8\" (UniqueName: \"kubernetes.io/projected/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-kube-api-access-dhxh8\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.495301 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.495227    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-workload-certs\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.495408 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.495311    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-workload-socket\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.495564 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.495542    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-credential-socket\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.495696 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.495586    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-data\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.495871 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.495846    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istiod-ca-cert\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.497392 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.497361    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-envoy\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.497776 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.497755    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-podinfo\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.503368 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.503344    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-token\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.503497 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.503480    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhxh8\" (UniqueName: \"kubernetes.io/projected/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-kube-api-access-dhxh8\") pod \"e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") " pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.651820 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.651733    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:20.942145 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.942065    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"]
Sep 14 03:10:20.945980 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:10:20.945953    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1b7ad87_3c3a_4dfd_b3f0_6a9e10a67d9e.slice/crio-2b66a19aa8319bc2066b31381ec3c9177652f45dec1f4599ebd861ae2460fcbd WatchSource:0}: Error finding container 2b66a19aa8319bc2066b31381ec3c9177652f45dec1f4599ebd861ae2460fcbd: Status 404 returned error can't find the container with id 2b66a19aa8319bc2066b31381ec3c9177652f45dec1f4599ebd861ae2460fcbd
Sep 14 03:10:20.948474 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.948442    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:10:20.948567 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.948504    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:10:20.948567 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:20.948550    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:10:21.914789 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:21.914743    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq" event={"ID":"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e","Type":"ContainerStarted","Data":"6f1aa3c1ec0b84b6b705a497a60654b59346407c815d60114d91c90d8452cec0"}
Sep 14 03:10:21.915162 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:21.914812    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq" event={"ID":"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e","Type":"ContainerStarted","Data":"2b66a19aa8319bc2066b31381ec3c9177652f45dec1f4599ebd861ae2460fcbd"}
Sep 14 03:10:21.941464 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:21.941394    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq" podStartSLOduration=1.9413789860000001 podStartE2EDuration="1.941378986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:10:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:10:21.940438066 +0000 UTC m=+1168.253461888" watchObservedRunningTime="2026-09-14 03:10:21.941378986 +0000 UTC m=+1168.254402804"
Sep 14 03:10:22.652742 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:22.652711    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:22.657708 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:22.657681    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:22.920022 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:22.919928    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:22.920990 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:22.920972    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:32.143094 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:32.143065    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:10:42.040108 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:42.040073    2574 generic.go:356] "Generic (PLEG): container finished" podID="d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" containerID="6f1aa3c1ec0b84b6b705a497a60654b59346407c815d60114d91c90d8452cec0" exitCode=0
Sep 14 03:10:42.040561 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:42.040147    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq" event={"ID":"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e","Type":"ContainerDied","Data":"6f1aa3c1ec0b84b6b705a497a60654b59346407c815d60114d91c90d8452cec0"}
Sep 14 03:10:42.040561 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:42.040490    2574 scope.go:117] "RemoveContainer" containerID="6f1aa3c1ec0b84b6b705a497a60654b59346407c815d60114d91c90d8452cec0"
Sep 14 03:10:42.041570 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:42.041539    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:10:42.041692 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:42.041626    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:10:42.041692 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:42.041672    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:10:43.047024 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:43.046989    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq" event={"ID":"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e","Type":"ContainerStarted","Data":"a6d02bed96cb55ac7b7e9c922f61a38762895a48286ce28712f860a3fd52c8b7"}
Sep 14 03:10:43.652589 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:43.652545    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:43.652798 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:43.652774    2574 patch_prober.go:28] interesting pod/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 03:10:43.652848 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:43.652827    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq" podUID="d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 03:10:44.652699 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:44.652666    2574 patch_prober.go:28] interesting pod/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 03:10:44.653087 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:44.652721    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq" podUID="d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 03:10:45.652466 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:45.652428    2574 patch_prober.go:28] interesting pod/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 03:10:45.652665 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:45.652502    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq" podUID="d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 03:10:46.652715 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:46.652680    2574 patch_prober.go:28] interesting pod/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 03:10:46.653101 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:46.652732    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq" podUID="d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 03:10:46.932368 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:46.932291    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:10:47.652285 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:47.652252    2574 patch_prober.go:28] interesting pod/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 03:10:47.652488 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:47.652309    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq" podUID="d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 03:10:48.655958 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:48.655922    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:48.656334 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:48.656204    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:48.656896 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:48.656876    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:10:49.542208 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.542178    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"]
Sep 14 03:10:49.542910 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.542891    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9b31752-5e13-47c7-992a-e79834207f29" containerName="istio-proxy"
Sep 14 03:10:49.546274 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.546255    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.548942 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.548923    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-9f77a9-openshift-default-dockercfg-82s74\""
Sep 14 03:10:49.558600 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.558579    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"]
Sep 14 03:10:49.657463 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.657429    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-credential-socket\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.657463 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.657464    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-envoy\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.657892 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.657487    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgvj5\" (UniqueName: \"kubernetes.io/projected/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-kube-api-access-jgvj5\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.657892 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.657545    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-workload-certs\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.657892 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.657566    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-podinfo\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.657892 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.657587    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-token\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.657892 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.657664    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-data\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.657892 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.657709    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istiod-ca-cert\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.657892 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.657728    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-workload-socket\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.759111 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.759072    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-data\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.759111 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.759120    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istiod-ca-cert\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.759339 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.759144    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-workload-socket\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.759339 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.759208    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-credential-socket\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.759339 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.759230    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-envoy\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.759339 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.759248    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgvj5\" (UniqueName: \"kubernetes.io/projected/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-kube-api-access-jgvj5\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.759339 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.759272    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-workload-certs\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.759339 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.759301    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-podinfo\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.759339 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.759332    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-token\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.759777 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.759555    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-data\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.759777 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.759583    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-credential-socket\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.759777 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.759656    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-workload-socket\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.759932 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.759912    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-workload-certs\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.759977 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.759958    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istiod-ca-cert\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.761791 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.761765    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-envoy\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.762077 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.762059    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-podinfo\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.768236 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.768191    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-token\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.768738 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.768717    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgvj5\" (UniqueName: \"kubernetes.io/projected/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-kube-api-access-jgvj5\") pod \"e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") " pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:49.856805 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:49.856742    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:50.132072 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:50.132049    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"]
Sep 14 03:10:50.133457 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:10:50.133424    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf9a5b9b_5dc8_4811_9091_5b8f8508244c.slice/crio-b44b4dde7ee11c66a0e097a92c80e4b103223a6aba30a371e1b98336ad8a9bdd WatchSource:0}: Error finding container b44b4dde7ee11c66a0e097a92c80e4b103223a6aba30a371e1b98336ad8a9bdd: Status 404 returned error can't find the container with id b44b4dde7ee11c66a0e097a92c80e4b103223a6aba30a371e1b98336ad8a9bdd
Sep 14 03:10:50.135625 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:50.135594    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:10:50.135694 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:50.135656    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:10:50.135694 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:50.135685    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:10:51.089335 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:51.089299    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl" event={"ID":"bf9a5b9b-5dc8-4811-9091-5b8f8508244c","Type":"ContainerStarted","Data":"450acff468342625292199e64178442371059fba5d118a4634c0188bc9d2cbe3"}
Sep 14 03:10:51.089843 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:51.089353    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl" event={"ID":"bf9a5b9b-5dc8-4811-9091-5b8f8508244c","Type":"ContainerStarted","Data":"b44b4dde7ee11c66a0e097a92c80e4b103223a6aba30a371e1b98336ad8a9bdd"}
Sep 14 03:10:51.112912 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:51.112860    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl" podStartSLOduration=2.112844429 podStartE2EDuration="2.112844429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:10:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:10:51.111587411 +0000 UTC m=+1197.424611229" watchObservedRunningTime="2026-09-14 03:10:51.112844429 +0000 UTC m=+1197.425868247"
Sep 14 03:10:51.857671 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:51.857640    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:51.862250 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:51.862226    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:52.094859 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:52.094834    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:52.095686 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:52.095666    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:10:54.187645 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:54.187619    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 03:10:54.194047 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:54.194026    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 03:10:54.216265 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:54.216247    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 03:10:54.222843 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:10:54.222826    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 03:11:02.687403 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:02.687369    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:11:13.213962 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:13.213922    2574 generic.go:356] "Generic (PLEG): container finished" podID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" containerID="450acff468342625292199e64178442371059fba5d118a4634c0188bc9d2cbe3" exitCode=0
Sep 14 03:11:13.214381 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:13.213969    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl" event={"ID":"bf9a5b9b-5dc8-4811-9091-5b8f8508244c","Type":"ContainerDied","Data":"450acff468342625292199e64178442371059fba5d118a4634c0188bc9d2cbe3"}
Sep 14 03:11:13.214381 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:13.214295    2574 scope.go:117] "RemoveContainer" containerID="450acff468342625292199e64178442371059fba5d118a4634c0188bc9d2cbe3"
Sep 14 03:11:13.215367 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:13.215335    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:11:13.215453 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:13.215405    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:11:13.215453 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:13.215436    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:11:14.220593 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:14.220559    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl" event={"ID":"bf9a5b9b-5dc8-4811-9091-5b8f8508244c","Type":"ContainerStarted","Data":"442d65f1e5039d54651c9462e411b8a908718efab86532deac41ac172958143c"}
Sep 14 03:11:14.857024 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:14.856993    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:11:14.857197 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:14.857161    2574 patch_prober.go:28] interesting pod/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 14 03:11:14.857242 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:14.857204    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl" podUID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 14 03:11:15.857619 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:15.857581    2574 patch_prober.go:28] interesting pod/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 14 03:11:15.858060 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:15.857642    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl" podUID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 14 03:11:16.858026 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:16.857992    2574 patch_prober.go:28] interesting pod/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 14 03:11:16.858404 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:16.858047    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl" podUID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 14 03:11:17.857610 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:17.857579    2574 patch_prober.go:28] interesting pod/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 14 03:11:17.857846 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:17.857634    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl" podUID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 14 03:11:18.754552 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:18.754502    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8993-a5c4f1"]
Sep 14 03:11:18.757988 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:18.757965    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-a5c4f1"
Sep 14 03:11:18.761403 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:18.761385    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4ff8v\""
Sep 14 03:11:18.770476 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:18.770454    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8993-a5c4f1"]
Sep 14 03:11:18.841573 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:18.841542    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjmzv\" (UniqueName: \"kubernetes.io/projected/d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6-kube-api-access-rjmzv\") pod \"test-curl-8993-a5c4f1\" (UID: \"d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6\") " pod="opendatahub/test-curl-8993-a5c4f1"
Sep 14 03:11:18.857651 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:18.857623    2574 patch_prober.go:28] interesting pod/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 14 03:11:18.857763 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:18.857673    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl" podUID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 14 03:11:18.942241 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:18.942210    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjmzv\" (UniqueName: \"kubernetes.io/projected/d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6-kube-api-access-rjmzv\") pod \"test-curl-8993-a5c4f1\" (UID: \"d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6\") " pod="opendatahub/test-curl-8993-a5c4f1"
Sep 14 03:11:18.951676 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:18.951653    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjmzv\" (UniqueName: \"kubernetes.io/projected/d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6-kube-api-access-rjmzv\") pod \"test-curl-8993-a5c4f1\" (UID: \"d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6\") " pod="opendatahub/test-curl-8993-a5c4f1"
Sep 14 03:11:19.068214 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:19.068106    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-a5c4f1"
Sep 14 03:11:19.154423 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:19.154359    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:11:19.345354 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:19.345266    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8993-a5c4f1"]
Sep 14 03:11:19.348233 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:11:19.348201    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd98dfc1c_c9a6_4579_bead_41cc3ae8d9d6.slice/crio-c41f6df14c4495034482a54f2d806fbfed377420d8586f79941fbd4cff271b5f WatchSource:0}: Error finding container c41f6df14c4495034482a54f2d806fbfed377420d8586f79941fbd4cff271b5f: Status 404 returned error can't find the container with id c41f6df14c4495034482a54f2d806fbfed377420d8586f79941fbd4cff271b5f
Sep 14 03:11:19.861276 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:19.861250    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:11:19.861671 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:19.861651    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:11:19.862360 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:19.862344    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:11:20.253344 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:20.253306    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8993-a5c4f1" event={"ID":"d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6","Type":"ContainerStarted","Data":"a9c5ec60151927a663ca4e737500ef416fa61367fc96cce7b9a621cef3c0810d"}
Sep 14 03:11:20.253344 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:20.253348    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8993-a5c4f1" event={"ID":"d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6","Type":"ContainerStarted","Data":"c41f6df14c4495034482a54f2d806fbfed377420d8586f79941fbd4cff271b5f"}
Sep 14 03:11:20.269756 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:20.269708    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8993-a5c4f1" podStartSLOduration=2.269694014 podStartE2EDuration="2.269694014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:11:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:11:20.268837996 +0000 UTC m=+1226.581861814" watchObservedRunningTime="2026-09-14 03:11:20.269694014 +0000 UTC m=+1226.582717833"
Sep 14 03:11:20.860024 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:20.859993    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8993-a5c4f1"]
Sep 14 03:11:20.867303 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:20.867273    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8993-a5c4f1"]
Sep 14 03:11:20.986825 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:20.986793    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8993-966098"]
Sep 14 03:11:20.987180 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:20.987167    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6" containerName="test-curl-8993-a5c4f1"
Sep 14 03:11:20.987223 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:20.987183    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6" containerName="test-curl-8993-a5c4f1"
Sep 14 03:11:20.987277 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:20.987266    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6" containerName="test-curl-8993-a5c4f1"
Sep 14 03:11:20.990338 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:20.990322    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-966098"
Sep 14 03:11:20.999188 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:20.999158    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8993-966098"]
Sep 14 03:11:21.060176 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:21.060152    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r59tm\" (UniqueName: \"kubernetes.io/projected/a71c8d03-6eea-419f-b212-e8a54fe0a202-kube-api-access-r59tm\") pod \"test-curl-8993-966098\" (UID: \"a71c8d03-6eea-419f-b212-e8a54fe0a202\") " pod="opendatahub/test-curl-8993-966098"
Sep 14 03:11:21.160984 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:21.160953    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r59tm\" (UniqueName: \"kubernetes.io/projected/a71c8d03-6eea-419f-b212-e8a54fe0a202-kube-api-access-r59tm\") pod \"test-curl-8993-966098\" (UID: \"a71c8d03-6eea-419f-b212-e8a54fe0a202\") " pod="opendatahub/test-curl-8993-966098"
Sep 14 03:11:21.171092 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:21.171062    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r59tm\" (UniqueName: \"kubernetes.io/projected/a71c8d03-6eea-419f-b212-e8a54fe0a202-kube-api-access-r59tm\") pod \"test-curl-8993-966098\" (UID: \"a71c8d03-6eea-419f-b212-e8a54fe0a202\") " pod="opendatahub/test-curl-8993-966098"
Sep 14 03:11:21.258065 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:21.258006    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8993-a5c4f1" container="test-curl-8993-a5c4f1"
Sep 14 03:11:21.300403 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:21.300382    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-966098"
Sep 14 03:11:21.572032 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:21.572005    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8993-966098"]
Sep 14 03:11:21.573681 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:11:21.573641    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda71c8d03_6eea_419f_b212_e8a54fe0a202.slice/crio-45ce1a200c605b581e351c9bd3f8726385f5d6427af995afcdcd85e39052978a WatchSource:0}: Error finding container 45ce1a200c605b581e351c9bd3f8726385f5d6427af995afcdcd85e39052978a: Status 404 returned error can't find the container with id 45ce1a200c605b581e351c9bd3f8726385f5d6427af995afcdcd85e39052978a
Sep 14 03:11:22.264343 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:22.264310    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8993-966098" event={"ID":"a71c8d03-6eea-419f-b212-e8a54fe0a202","Type":"ContainerStarted","Data":"5a842b3fa580f8117096cb00660a16e19120b071e8ac043c82341055668ff01d"}
Sep 14 03:11:22.264761 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:22.264353    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8993-966098" event={"ID":"a71c8d03-6eea-419f-b212-e8a54fe0a202","Type":"ContainerStarted","Data":"45ce1a200c605b581e351c9bd3f8726385f5d6427af995afcdcd85e39052978a"}
Sep 14 03:11:22.264761 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:22.264443    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8993-a5c4f1" podUID="d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6" containerName="test-curl-8993-a5c4f1" containerID="cri-o://a9c5ec60151927a663ca4e737500ef416fa61367fc96cce7b9a621cef3c0810d" gracePeriod=2
Sep 14 03:11:22.280899 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:22.280854    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8993-966098" podStartSLOduration=2.280838954 podStartE2EDuration="2.280838954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:11:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:11:22.280276393 +0000 UTC m=+1228.593300211" watchObservedRunningTime="2026-09-14 03:11:22.280838954 +0000 UTC m=+1228.593862808"
Sep 14 03:11:22.930891 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:22.930805    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8993-966098"]
Sep 14 03:11:22.936053 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:22.936022    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8993-966098"]
Sep 14 03:11:23.056194 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:23.056169    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8993-15027f"]
Sep 14 03:11:23.056606 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:23.056590    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a71c8d03-6eea-419f-b212-e8a54fe0a202" containerName="test-curl-8993-966098"
Sep 14 03:11:23.056662 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:23.056610    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a71c8d03-6eea-419f-b212-e8a54fe0a202" containerName="test-curl-8993-966098"
Sep 14 03:11:23.056699 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:23.056686    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a71c8d03-6eea-419f-b212-e8a54fe0a202" containerName="test-curl-8993-966098"
Sep 14 03:11:23.059708 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:23.059691    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-15027f"
Sep 14 03:11:23.066904 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:23.066880    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8993-15027f"]
Sep 14 03:11:23.179202 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:23.179175    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99xlc\" (UniqueName: \"kubernetes.io/projected/13991953-6cdb-467e-b500-f3eeaaa38e15-kube-api-access-99xlc\") pod \"test-curl-8993-15027f\" (UID: \"13991953-6cdb-467e-b500-f3eeaaa38e15\") " pod="opendatahub/test-curl-8993-15027f"
Sep 14 03:11:23.269280 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:23.269253    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8993-966098" container="test-curl-8993-966098"
Sep 14 03:11:23.280497 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:23.280480    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99xlc\" (UniqueName: \"kubernetes.io/projected/13991953-6cdb-467e-b500-f3eeaaa38e15-kube-api-access-99xlc\") pod \"test-curl-8993-15027f\" (UID: \"13991953-6cdb-467e-b500-f3eeaaa38e15\") " pod="opendatahub/test-curl-8993-15027f"
Sep 14 03:11:23.290581 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:23.290563    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99xlc\" (UniqueName: \"kubernetes.io/projected/13991953-6cdb-467e-b500-f3eeaaa38e15-kube-api-access-99xlc\") pod \"test-curl-8993-15027f\" (UID: \"13991953-6cdb-467e-b500-f3eeaaa38e15\") " pod="opendatahub/test-curl-8993-15027f"
Sep 14 03:11:23.370644 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:23.370617    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-15027f"
Sep 14 03:11:23.648616 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:23.648590    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8993-15027f"]
Sep 14 03:11:23.650251 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:11:23.650225    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13991953_6cdb_467e_b500_f3eeaaa38e15.slice/crio-1f9e035c35c7056dac50fafe21266074d008d8bba4528e866bd87ef471fd56ee WatchSource:0}: Error finding container 1f9e035c35c7056dac50fafe21266074d008d8bba4528e866bd87ef471fd56ee: Status 404 returned error can't find the container with id 1f9e035c35c7056dac50fafe21266074d008d8bba4528e866bd87ef471fd56ee
Sep 14 03:11:24.277060 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:24.277033    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8993-15027f" event={"ID":"13991953-6cdb-467e-b500-f3eeaaa38e15","Type":"ContainerStarted","Data":"dae1c5ea7988af15f41ac2910859525562d9aa304939530cde2cc8186b97281c"}
Sep 14 03:11:24.277372 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:24.277070    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8993-15027f" event={"ID":"13991953-6cdb-467e-b500-f3eeaaa38e15","Type":"ContainerStarted","Data":"1f9e035c35c7056dac50fafe21266074d008d8bba4528e866bd87ef471fd56ee"}
Sep 14 03:11:24.277372 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:24.277245    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8993-966098" podUID="a71c8d03-6eea-419f-b212-e8a54fe0a202" containerName="test-curl-8993-966098" containerID="cri-o://5a842b3fa580f8117096cb00660a16e19120b071e8ac043c82341055668ff01d" gracePeriod=2
Sep 14 03:11:24.295915 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:24.295865    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8993-15027f" podStartSLOduration=1.295848906 podStartE2EDuration="1.295848906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:11:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:11:24.294281581 +0000 UTC m=+1230.607305397" watchObservedRunningTime="2026-09-14 03:11:24.295848906 +0000 UTC m=+1230.608872725"
Sep 14 03:11:24.377047 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:24.377024    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-a5c4f1"
Sep 14 03:11:24.494673 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:24.494643    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjmzv\" (UniqueName: \"kubernetes.io/projected/d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6-kube-api-access-rjmzv\") pod \"d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6\" (UID: \"d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6\") "
Sep 14 03:11:24.497669 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:24.497639    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6-kube-api-access-rjmzv" (OuterVolumeSpecName: "kube-api-access-rjmzv") pod "d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6" (UID: "d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6"). InnerVolumeSpecName "kube-api-access-rjmzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:11:24.595702 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:24.595649    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjmzv\" (UniqueName: \"kubernetes.io/projected/d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6-kube-api-access-rjmzv\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:24.883752 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:24.883671    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8993-15027f"]
Sep 14 03:11:24.887439 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:24.887414    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8993-15027f"]
Sep 14 03:11:25.001309 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.001277    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8993-e81ec8"]
Sep 14 03:11:25.001705 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.001691    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13991953-6cdb-467e-b500-f3eeaaa38e15" containerName="test-curl-8993-15027f"
Sep 14 03:11:25.001747 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.001708    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="13991953-6cdb-467e-b500-f3eeaaa38e15" containerName="test-curl-8993-15027f"
Sep 14 03:11:25.001785 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.001763    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="13991953-6cdb-467e-b500-f3eeaaa38e15" containerName="test-curl-8993-15027f"
Sep 14 03:11:25.004716 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.004699    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-e81ec8"
Sep 14 03:11:25.010503 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.010480    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8993-e81ec8"]
Sep 14 03:11:25.100499 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.100473    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6jw4\" (UniqueName: \"kubernetes.io/projected/b0d82ae7-1307-4213-8f52-f89a81002198-kube-api-access-k6jw4\") pod \"test-curl-8993-e81ec8\" (UID: \"b0d82ae7-1307-4213-8f52-f89a81002198\") " pod="opendatahub/test-curl-8993-e81ec8"
Sep 14 03:11:25.201728 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.201656    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6jw4\" (UniqueName: \"kubernetes.io/projected/b0d82ae7-1307-4213-8f52-f89a81002198-kube-api-access-k6jw4\") pod \"test-curl-8993-e81ec8\" (UID: \"b0d82ae7-1307-4213-8f52-f89a81002198\") " pod="opendatahub/test-curl-8993-e81ec8"
Sep 14 03:11:25.218070 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.218049    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6jw4\" (UniqueName: \"kubernetes.io/projected/b0d82ae7-1307-4213-8f52-f89a81002198-kube-api-access-k6jw4\") pod \"test-curl-8993-e81ec8\" (UID: \"b0d82ae7-1307-4213-8f52-f89a81002198\") " pod="opendatahub/test-curl-8993-e81ec8"
Sep 14 03:11:25.283716 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.283687    2574 generic.go:356] "Generic (PLEG): container finished" podID="d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6" containerID="a9c5ec60151927a663ca4e737500ef416fa61367fc96cce7b9a621cef3c0810d" exitCode=137
Sep 14 03:11:25.284039 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.283736    2574 scope.go:117] "RemoveContainer" containerID="a9c5ec60151927a663ca4e737500ef416fa61367fc96cce7b9a621cef3c0810d"
Sep 14 03:11:25.284039 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.283743    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-a5c4f1"
Sep 14 03:11:25.284140 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:25.284099    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8993-15027f" container="test-curl-8993-15027f"
Sep 14 03:11:25.292337 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.292318    2574 scope.go:117] "RemoveContainer" containerID="a9c5ec60151927a663ca4e737500ef416fa61367fc96cce7b9a621cef3c0810d"
Sep 14 03:11:25.292635 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:25.292617    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9c5ec60151927a663ca4e737500ef416fa61367fc96cce7b9a621cef3c0810d\": container with ID starting with a9c5ec60151927a663ca4e737500ef416fa61367fc96cce7b9a621cef3c0810d not found: ID does not exist" containerID="a9c5ec60151927a663ca4e737500ef416fa61367fc96cce7b9a621cef3c0810d"
Sep 14 03:11:25.292708 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.292647    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9c5ec60151927a663ca4e737500ef416fa61367fc96cce7b9a621cef3c0810d"} err="failed to get container status \"a9c5ec60151927a663ca4e737500ef416fa61367fc96cce7b9a621cef3c0810d\": rpc error: code = NotFound desc = could not find container \"a9c5ec60151927a663ca4e737500ef416fa61367fc96cce7b9a621cef3c0810d\": container with ID starting with a9c5ec60151927a663ca4e737500ef416fa61367fc96cce7b9a621cef3c0810d not found: ID does not exist"
Sep 14 03:11:25.294476 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:25.294455    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8993-a5c4f1\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" podUID="d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6" pod="opendatahub/test-curl-8993-a5c4f1"
Sep 14 03:11:25.314805 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.314787    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-e81ec8"
Sep 14 03:11:25.593706 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:25.593684    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8993-e81ec8"]
Sep 14 03:11:25.595626 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:11:25.595597    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0d82ae7_1307_4213_8f52_f89a81002198.slice/crio-c03c538d83280f4353b6277c54a4385ca7c07498d56197350ea30d18145566bc WatchSource:0}: Error finding container c03c538d83280f4353b6277c54a4385ca7c07498d56197350ea30d18145566bc: Status 404 returned error can't find the container with id c03c538d83280f4353b6277c54a4385ca7c07498d56197350ea30d18145566bc
Sep 14 03:11:26.216531 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:26.216482    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6" path="/var/lib/kubelet/pods/d98dfc1c-c9a6-4579-bead-41cc3ae8d9d6/volumes"
Sep 14 03:11:26.292359 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:26.292331    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8993-e81ec8" event={"ID":"b0d82ae7-1307-4213-8f52-f89a81002198","Type":"ContainerStarted","Data":"3e81ff5083cfbc9f37ee9afffc53ba414b1fb21f34bc1310bf3b300298eab38e"}
Sep 14 03:11:26.292712 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:26.292374    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8993-e81ec8" event={"ID":"b0d82ae7-1307-4213-8f52-f89a81002198","Type":"ContainerStarted","Data":"c03c538d83280f4353b6277c54a4385ca7c07498d56197350ea30d18145566bc"}
Sep 14 03:11:26.292712 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:26.292390    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8993-15027f" podUID="13991953-6cdb-467e-b500-f3eeaaa38e15" containerName="test-curl-8993-15027f" containerID="cri-o://dae1c5ea7988af15f41ac2910859525562d9aa304939530cde2cc8186b97281c" gracePeriod=2
Sep 14 03:11:26.310066 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:26.310029    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8993-e81ec8" podStartSLOduration=2.310015578 podStartE2EDuration="2.310015578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:11:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:11:26.308273865 +0000 UTC m=+1232.621297684" watchObservedRunningTime="2026-09-14 03:11:26.310015578 +0000 UTC m=+1232.623039409"
Sep 14 03:11:26.450454 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:26.450428    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-966098"
Sep 14 03:11:26.514433 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:26.514413    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r59tm\" (UniqueName: \"kubernetes.io/projected/a71c8d03-6eea-419f-b212-e8a54fe0a202-kube-api-access-r59tm\") pod \"a71c8d03-6eea-419f-b212-e8a54fe0a202\" (UID: \"a71c8d03-6eea-419f-b212-e8a54fe0a202\") "
Sep 14 03:11:26.516495 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:26.516476    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a71c8d03-6eea-419f-b212-e8a54fe0a202-kube-api-access-r59tm" (OuterVolumeSpecName: "kube-api-access-r59tm") pod "a71c8d03-6eea-419f-b212-e8a54fe0a202" (UID: "a71c8d03-6eea-419f-b212-e8a54fe0a202"). InnerVolumeSpecName "kube-api-access-r59tm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:11:26.615881 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:26.615856    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r59tm\" (UniqueName: \"kubernetes.io/projected/a71c8d03-6eea-419f-b212-e8a54fe0a202-kube-api-access-r59tm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:26.876237 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:26.876100    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8993-e81ec8"]
Sep 14 03:11:26.878443 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:26.878417    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8993-e81ec8"]
Sep 14 03:11:27.298291 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.298256    2574 generic.go:356] "Generic (PLEG): container finished" podID="a71c8d03-6eea-419f-b212-e8a54fe0a202" containerID="5a842b3fa580f8117096cb00660a16e19120b071e8ac043c82341055668ff01d" exitCode=137
Sep 14 03:11:27.298291 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.298290    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-966098"
Sep 14 03:11:27.298750 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.298356    2574 scope.go:117] "RemoveContainer" containerID="5a842b3fa580f8117096cb00660a16e19120b071e8ac043c82341055668ff01d"
Sep 14 03:11:27.298750 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:27.298586    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8993-e81ec8" container="test-curl-8993-e81ec8"
Sep 14 03:11:27.307958 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.307943    2574 scope.go:117] "RemoveContainer" containerID="5a842b3fa580f8117096cb00660a16e19120b071e8ac043c82341055668ff01d"
Sep 14 03:11:27.308234 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:27.308216    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a842b3fa580f8117096cb00660a16e19120b071e8ac043c82341055668ff01d\": container with ID starting with 5a842b3fa580f8117096cb00660a16e19120b071e8ac043c82341055668ff01d not found: ID does not exist" containerID="5a842b3fa580f8117096cb00660a16e19120b071e8ac043c82341055668ff01d"
Sep 14 03:11:27.308295 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.308247    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a842b3fa580f8117096cb00660a16e19120b071e8ac043c82341055668ff01d"} err="failed to get container status \"5a842b3fa580f8117096cb00660a16e19120b071e8ac043c82341055668ff01d\": rpc error: code = NotFound desc = could not find container \"5a842b3fa580f8117096cb00660a16e19120b071e8ac043c82341055668ff01d\": container with ID starting with 5a842b3fa580f8117096cb00660a16e19120b071e8ac043c82341055668ff01d not found: ID does not exist"
Sep 14 03:11:27.308920 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:27.308899    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8993-966098\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" podUID="a71c8d03-6eea-419f-b212-e8a54fe0a202" pod="opendatahub/test-curl-8993-966098"
Sep 14 03:11:27.749662 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.749627    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"]
Sep 14 03:11:27.749966 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.749954    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0d82ae7-1307-4213-8f52-f89a81002198" containerName="test-curl-8993-e81ec8"
Sep 14 03:11:27.750005 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.749968    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0d82ae7-1307-4213-8f52-f89a81002198" containerName="test-curl-8993-e81ec8"
Sep 14 03:11:27.750043 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.750035    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0d82ae7-1307-4213-8f52-f89a81002198" containerName="test-curl-8993-e81ec8"
Sep 14 03:11:27.754149 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.754133    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.757747 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.757727    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d6ac6dc8-openshift-default-dockercfg-tbmq9\""
Sep 14 03:11:27.778782 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.778749    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"]
Sep 14 03:11:27.818912 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:27.818883    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8993-966098\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" podUID="a71c8d03-6eea-419f-b212-e8a54fe0a202" pod="opendatahub/test-curl-8993-966098"
Sep 14 03:11:27.828289 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.828261    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-data\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.828385 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.828330    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3bf449c-92d9-42a0-a589-85fb9862e51a-istiod-ca-cert\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.828385 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.828372    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-envoy\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.828495 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.828399    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-workload-certs\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.828495 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.828458    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-credential-socket\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.828642 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.828506    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-podinfo\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.828642 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.828591    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpp44\" (UniqueName: \"kubernetes.io/projected/b3bf449c-92d9-42a0-a589-85fb9862e51a-kube-api-access-cpp44\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.828744 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.828642    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-workload-socket\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.828744 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.828681    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-token\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.929720 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.929685    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3bf449c-92d9-42a0-a589-85fb9862e51a-istiod-ca-cert\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.929720 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.929727    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-envoy\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.929971 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.929748    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-workload-certs\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.929971 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.929870    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-credential-socket\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.929971 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.929907    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-podinfo\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.929971 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.929946    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpp44\" (UniqueName: \"kubernetes.io/projected/b3bf449c-92d9-42a0-a589-85fb9862e51a-kube-api-access-cpp44\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.930171 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.930004    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-workload-socket\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.930171 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.930047    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-token\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.930171 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.930102    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-data\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.930171 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.930115    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-workload-certs\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.930373 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.930207    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-credential-socket\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.930458 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.930436    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-data\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.930682 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.930493    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-workload-socket\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.930682 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.930549    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3bf449c-92d9-42a0-a589-85fb9862e51a-istiod-ca-cert\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.932230 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.932199    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-envoy\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.932553 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.932534    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-podinfo\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.939799 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.939778    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpp44\" (UniqueName: \"kubernetes.io/projected/b3bf449c-92d9-42a0-a589-85fb9862e51a-kube-api-access-cpp44\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:27.940126 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:27.940105    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-token\") pod \"e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") " pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:28.064950 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:28.064873    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:28.215762 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:28.215733    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a71c8d03-6eea-419f-b212-e8a54fe0a202" path="/var/lib/kubelet/pods/a71c8d03-6eea-419f-b212-e8a54fe0a202/volumes"
Sep 14 03:11:28.305548 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:28.305491    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8993-e81ec8" podUID="b0d82ae7-1307-4213-8f52-f89a81002198" containerName="test-curl-8993-e81ec8" containerID="cri-o://3e81ff5083cfbc9f37ee9afffc53ba414b1fb21f34bc1310bf3b300298eab38e" gracePeriod=2
Sep 14 03:11:28.349501 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:28.349090    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"]
Sep 14 03:11:28.358645 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:11:28.358610    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3bf449c_92d9_42a0_a589_85fb9862e51a.slice/crio-564655d6fc900ef542ab3768a935e63a3586114be882a12c4feca41955875fea WatchSource:0}: Error finding container 564655d6fc900ef542ab3768a935e63a3586114be882a12c4feca41955875fea: Status 404 returned error can't find the container with id 564655d6fc900ef542ab3768a935e63a3586114be882a12c4feca41955875fea
Sep 14 03:11:28.361490 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:28.361437    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:11:28.361717 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:28.361688    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:11:28.361766 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:28.361752    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:11:28.489710 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:28.489687    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-15027f"
Sep 14 03:11:28.637831 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:28.637749    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99xlc\" (UniqueName: \"kubernetes.io/projected/13991953-6cdb-467e-b500-f3eeaaa38e15-kube-api-access-99xlc\") pod \"13991953-6cdb-467e-b500-f3eeaaa38e15\" (UID: \"13991953-6cdb-467e-b500-f3eeaaa38e15\") "
Sep 14 03:11:28.639936 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:28.639899    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13991953-6cdb-467e-b500-f3eeaaa38e15-kube-api-access-99xlc" (OuterVolumeSpecName: "kube-api-access-99xlc") pod "13991953-6cdb-467e-b500-f3eeaaa38e15" (UID: "13991953-6cdb-467e-b500-f3eeaaa38e15"). InnerVolumeSpecName "kube-api-access-99xlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:11:28.739391 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:28.739362    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99xlc\" (UniqueName: \"kubernetes.io/projected/13991953-6cdb-467e-b500-f3eeaaa38e15-kube-api-access-99xlc\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:29.311828 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:29.311789    2574 generic.go:356] "Generic (PLEG): container finished" podID="13991953-6cdb-467e-b500-f3eeaaa38e15" containerID="dae1c5ea7988af15f41ac2910859525562d9aa304939530cde2cc8186b97281c" exitCode=137
Sep 14 03:11:29.312276 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:29.311831    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-15027f"
Sep 14 03:11:29.312276 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:29.311879    2574 scope.go:117] "RemoveContainer" containerID="dae1c5ea7988af15f41ac2910859525562d9aa304939530cde2cc8186b97281c"
Sep 14 03:11:29.313739 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:29.313712    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz" event={"ID":"b3bf449c-92d9-42a0-a589-85fb9862e51a","Type":"ContainerStarted","Data":"bb0037605705cd19aa640c220f1001e0e2caea74aa9f4a59586b741bb3fe4e5b"}
Sep 14 03:11:29.313860 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:29.313768    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz" event={"ID":"b3bf449c-92d9-42a0-a589-85fb9862e51a","Type":"ContainerStarted","Data":"564655d6fc900ef542ab3768a935e63a3586114be882a12c4feca41955875fea"}
Sep 14 03:11:29.320913 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:29.320895    2574 scope.go:117] "RemoveContainer" containerID="dae1c5ea7988af15f41ac2910859525562d9aa304939530cde2cc8186b97281c"
Sep 14 03:11:29.321182 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:29.321163    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dae1c5ea7988af15f41ac2910859525562d9aa304939530cde2cc8186b97281c\": container with ID starting with dae1c5ea7988af15f41ac2910859525562d9aa304939530cde2cc8186b97281c not found: ID does not exist" containerID="dae1c5ea7988af15f41ac2910859525562d9aa304939530cde2cc8186b97281c"
Sep 14 03:11:29.321245 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:29.321189    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dae1c5ea7988af15f41ac2910859525562d9aa304939530cde2cc8186b97281c"} err="failed to get container status \"dae1c5ea7988af15f41ac2910859525562d9aa304939530cde2cc8186b97281c\": rpc error: code = NotFound desc = could not find container \"dae1c5ea7988af15f41ac2910859525562d9aa304939530cde2cc8186b97281c\": container with ID starting with dae1c5ea7988af15f41ac2910859525562d9aa304939530cde2cc8186b97281c not found: ID does not exist"
Sep 14 03:11:29.339417 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:29.339380    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz" podStartSLOduration=2.339367381 podStartE2EDuration="2.339367381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:11:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:11:29.336604521 +0000 UTC m=+1235.649628341" watchObservedRunningTime="2026-09-14 03:11:29.339367381 +0000 UTC m=+1235.652391261"
Sep 14 03:11:29.339679 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:29.339651    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8993-15027f\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" podUID="13991953-6cdb-467e-b500-f3eeaaa38e15" pod="opendatahub/test-curl-8993-15027f"
Sep 14 03:11:30.065489 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.065453    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:30.070127 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.070107    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:30.111615 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:30.111588    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8993-15027f\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" podUID="13991953-6cdb-467e-b500-f3eeaaa38e15" pod="opendatahub/test-curl-8993-15027f"
Sep 14 03:11:30.188895 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.188871    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8996-5eebf6"]
Sep 14 03:11:30.193324 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.193309    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8996-5eebf6"
Sep 14 03:11:30.218290 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.218260    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13991953-6cdb-467e-b500-f3eeaaa38e15" path="/var/lib/kubelet/pods/13991953-6cdb-467e-b500-f3eeaaa38e15/volumes"
Sep 14 03:11:30.218684 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.218661    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8996-5eebf6"]
Sep 14 03:11:30.232379 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:30.232356    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8993-15027f\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" podUID="13991953-6cdb-467e-b500-f3eeaaa38e15" pod="opendatahub/test-curl-8993-15027f"
Sep 14 03:11:30.254818 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.254794    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvjcb\" (UniqueName: \"kubernetes.io/projected/ce90edac-2863-437f-a37f-d9b84d939c0c-kube-api-access-wvjcb\") pod \"test-curl-8996-5eebf6\" (UID: \"ce90edac-2863-437f-a37f-d9b84d939c0c\") " pod="opendatahub/test-curl-8996-5eebf6"
Sep 14 03:11:30.320139 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.320121    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:30.321019 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.321002    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:30.355286 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.355267    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvjcb\" (UniqueName: \"kubernetes.io/projected/ce90edac-2863-437f-a37f-d9b84d939c0c-kube-api-access-wvjcb\") pod \"test-curl-8996-5eebf6\" (UID: \"ce90edac-2863-437f-a37f-d9b84d939c0c\") " pod="opendatahub/test-curl-8996-5eebf6"
Sep 14 03:11:30.374406 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.374385    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvjcb\" (UniqueName: \"kubernetes.io/projected/ce90edac-2863-437f-a37f-d9b84d939c0c-kube-api-access-wvjcb\") pod \"test-curl-8996-5eebf6\" (UID: \"ce90edac-2863-437f-a37f-d9b84d939c0c\") " pod="opendatahub/test-curl-8996-5eebf6"
Sep 14 03:11:30.502281 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.502257    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8996-5eebf6"
Sep 14 03:11:30.502394 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.502300    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-e81ec8"
Sep 14 03:11:30.659205 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.659137    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6jw4\" (UniqueName: \"kubernetes.io/projected/b0d82ae7-1307-4213-8f52-f89a81002198-kube-api-access-k6jw4\") pod \"b0d82ae7-1307-4213-8f52-f89a81002198\" (UID: \"b0d82ae7-1307-4213-8f52-f89a81002198\") "
Sep 14 03:11:30.661424 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.661399    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0d82ae7-1307-4213-8f52-f89a81002198-kube-api-access-k6jw4" (OuterVolumeSpecName: "kube-api-access-k6jw4") pod "b0d82ae7-1307-4213-8f52-f89a81002198" (UID: "b0d82ae7-1307-4213-8f52-f89a81002198"). InnerVolumeSpecName "kube-api-access-k6jw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:11:30.759883 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.759854    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6jw4\" (UniqueName: \"kubernetes.io/projected/b0d82ae7-1307-4213-8f52-f89a81002198-kube-api-access-k6jw4\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:30.777017 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:30.776993    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8996-5eebf6"]
Sep 14 03:11:30.778764 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:11:30.778739    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce90edac_2863_437f_a37f_d9b84d939c0c.slice/crio-b225e6fbff7c04917f215d5d53ed18cd91d4dc560f94da9fd65274f5b08fb29e WatchSource:0}: Error finding container b225e6fbff7c04917f215d5d53ed18cd91d4dc560f94da9fd65274f5b08fb29e: Status 404 returned error can't find the container with id b225e6fbff7c04917f215d5d53ed18cd91d4dc560f94da9fd65274f5b08fb29e
Sep 14 03:11:31.327086 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:31.327049    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8996-5eebf6" event={"ID":"ce90edac-2863-437f-a37f-d9b84d939c0c","Type":"ContainerStarted","Data":"bf2709884391c2e3555fb6db56d4fbc3c19941072ef9575745c75466a5a887db"}
Sep 14 03:11:31.327086 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:31.327089    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8996-5eebf6" event={"ID":"ce90edac-2863-437f-a37f-d9b84d939c0c","Type":"ContainerStarted","Data":"b225e6fbff7c04917f215d5d53ed18cd91d4dc560f94da9fd65274f5b08fb29e"}
Sep 14 03:11:31.328317 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:31.328290    2574 generic.go:356] "Generic (PLEG): container finished" podID="b0d82ae7-1307-4213-8f52-f89a81002198" containerID="3e81ff5083cfbc9f37ee9afffc53ba414b1fb21f34bc1310bf3b300298eab38e" exitCode=137
Sep 14 03:11:31.328439 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:31.328327    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8993-e81ec8"
Sep 14 03:11:31.328439 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:31.328382    2574 scope.go:117] "RemoveContainer" containerID="3e81ff5083cfbc9f37ee9afffc53ba414b1fb21f34bc1310bf3b300298eab38e"
Sep 14 03:11:31.338003 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:31.337985    2574 scope.go:117] "RemoveContainer" containerID="3e81ff5083cfbc9f37ee9afffc53ba414b1fb21f34bc1310bf3b300298eab38e"
Sep 14 03:11:31.338286 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:31.338263    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e81ff5083cfbc9f37ee9afffc53ba414b1fb21f34bc1310bf3b300298eab38e\": container with ID starting with 3e81ff5083cfbc9f37ee9afffc53ba414b1fb21f34bc1310bf3b300298eab38e not found: ID does not exist" containerID="3e81ff5083cfbc9f37ee9afffc53ba414b1fb21f34bc1310bf3b300298eab38e"
Sep 14 03:11:31.338359 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:31.338297    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e81ff5083cfbc9f37ee9afffc53ba414b1fb21f34bc1310bf3b300298eab38e"} err="failed to get container status \"3e81ff5083cfbc9f37ee9afffc53ba414b1fb21f34bc1310bf3b300298eab38e\": rpc error: code = NotFound desc = could not find container \"3e81ff5083cfbc9f37ee9afffc53ba414b1fb21f34bc1310bf3b300298eab38e\": container with ID starting with 3e81ff5083cfbc9f37ee9afffc53ba414b1fb21f34bc1310bf3b300298eab38e not found: ID does not exist"
Sep 14 03:11:31.377376 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:31.377325    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8996-5eebf6" podStartSLOduration=1.377311125 podStartE2EDuration="1.377311125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:11:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:11:31.37574877 +0000 UTC m=+1237.688772591" watchObservedRunningTime="2026-09-14 03:11:31.377311125 +0000 UTC m=+1237.690334944"
Sep 14 03:11:31.378230 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:31.378203    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8993-e81ec8\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" podUID="b0d82ae7-1307-4213-8f52-f89a81002198" pod="opendatahub/test-curl-8993-e81ec8"
Sep 14 03:11:32.036914 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:32.036884    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8996-5eebf6"]
Sep 14 03:11:32.047267 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:32.047239    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8996-5eebf6"]
Sep 14 03:11:32.221450 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:32.221370    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0d82ae7-1307-4213-8f52-f89a81002198" path="/var/lib/kubelet/pods/b0d82ae7-1307-4213-8f52-f89a81002198/volumes"
Sep 14 03:11:32.334596 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:32.334571    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8996-5eebf6" container="test-curl-8996-5eebf6"
Sep 14 03:11:33.340036 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:33.339999    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8996-5eebf6" podUID="ce90edac-2863-437f-a37f-d9b84d939c0c" containerName="test-curl-8996-5eebf6" containerID="cri-o://bf2709884391c2e3555fb6db56d4fbc3c19941072ef9575745c75466a5a887db" gracePeriod=2
Sep 14 03:11:33.716392 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:33.716317    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:11:33.742277 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:33.742250    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"]
Sep 14 03:11:34.580270 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:34.580236    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"]
Sep 14 03:11:35.488444 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:35.488423    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8996-5eebf6"
Sep 14 03:11:35.603113 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:35.603057    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvjcb\" (UniqueName: \"kubernetes.io/projected/ce90edac-2863-437f-a37f-d9b84d939c0c-kube-api-access-wvjcb\") pod \"ce90edac-2863-437f-a37f-d9b84d939c0c\" (UID: \"ce90edac-2863-437f-a37f-d9b84d939c0c\") "
Sep 14 03:11:35.605164 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:35.605142    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce90edac-2863-437f-a37f-d9b84d939c0c-kube-api-access-wvjcb" (OuterVolumeSpecName: "kube-api-access-wvjcb") pod "ce90edac-2863-437f-a37f-d9b84d939c0c" (UID: "ce90edac-2863-437f-a37f-d9b84d939c0c"). InnerVolumeSpecName "kube-api-access-wvjcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:11:35.704882 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:35.704858    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvjcb\" (UniqueName: \"kubernetes.io/projected/ce90edac-2863-437f-a37f-d9b84d939c0c-kube-api-access-wvjcb\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:36.215472 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:36.215437    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce90edac-2863-437f-a37f-d9b84d939c0c" path="/var/lib/kubelet/pods/ce90edac-2863-437f-a37f-d9b84d939c0c/volumes"
Sep 14 03:11:36.356420 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:36.356389    2574 generic.go:356] "Generic (PLEG): container finished" podID="ce90edac-2863-437f-a37f-d9b84d939c0c" containerID="bf2709884391c2e3555fb6db56d4fbc3c19941072ef9575745c75466a5a887db" exitCode=137
Sep 14 03:11:36.356604 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:36.356433    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8996-5eebf6"
Sep 14 03:11:36.356604 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:36.356477    2574 scope.go:117] "RemoveContainer" containerID="bf2709884391c2e3555fb6db56d4fbc3c19941072ef9575745c75466a5a887db"
Sep 14 03:11:36.367915 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:36.367894    2574 scope.go:117] "RemoveContainer" containerID="bf2709884391c2e3555fb6db56d4fbc3c19941072ef9575745c75466a5a887db"
Sep 14 03:11:36.368192 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:36.368173    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf2709884391c2e3555fb6db56d4fbc3c19941072ef9575745c75466a5a887db\": container with ID starting with bf2709884391c2e3555fb6db56d4fbc3c19941072ef9575745c75466a5a887db not found: ID does not exist" containerID="bf2709884391c2e3555fb6db56d4fbc3c19941072ef9575745c75466a5a887db"
Sep 14 03:11:36.368283 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:36.368200    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf2709884391c2e3555fb6db56d4fbc3c19941072ef9575745c75466a5a887db"} err="failed to get container status \"bf2709884391c2e3555fb6db56d4fbc3c19941072ef9575745c75466a5a887db\": rpc error: code = NotFound desc = could not find container \"bf2709884391c2e3555fb6db56d4fbc3c19941072ef9575745c75466a5a887db\": container with ID starting with bf2709884391c2e3555fb6db56d4fbc3c19941072ef9575745c75466a5a887db not found: ID does not exist"
Sep 14 03:11:47.736710 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:47.736631    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:11:48.761052 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:48.760997    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp" podUID="9b6ef270-f2f5-48ac-8b8a-8e4297318330" containerName="main" containerID="cri-o://5b33497d5f5d58c81158ca8c6cc50572cb5909b70328502e5de24191175db777" gracePeriod=45
Sep 14 03:11:49.041797 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.041775    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:11:49.131733 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.131694    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-tmp-dir\") pod \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") "
Sep 14 03:11:49.131733 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.131739    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6zqh\" (UniqueName: \"kubernetes.io/projected/9b6ef270-f2f5-48ac-8b8a-8e4297318330-kube-api-access-g6zqh\") pod \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") "
Sep 14 03:11:49.132004 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.131757    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-home\") pod \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") "
Sep 14 03:11:49.132004 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.131812    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-model-cache\") pod \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") "
Sep 14 03:11:49.132126 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.132013    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9b6ef270-f2f5-48ac-8b8a-8e4297318330" (UID: "9b6ef270-f2f5-48ac-8b8a-8e4297318330"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:11:49.132126 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.132070    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-home" (OuterVolumeSpecName: "home") pod "9b6ef270-f2f5-48ac-8b8a-8e4297318330" (UID: "9b6ef270-f2f5-48ac-8b8a-8e4297318330"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:11:49.132241 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.132169    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-model-cache" (OuterVolumeSpecName: "model-cache") pod "9b6ef270-f2f5-48ac-8b8a-8e4297318330" (UID: "9b6ef270-f2f5-48ac-8b8a-8e4297318330"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:11:49.134116 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.134093    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b6ef270-f2f5-48ac-8b8a-8e4297318330-kube-api-access-g6zqh" (OuterVolumeSpecName: "kube-api-access-g6zqh") pod "9b6ef270-f2f5-48ac-8b8a-8e4297318330" (UID: "9b6ef270-f2f5-48ac-8b8a-8e4297318330"). InnerVolumeSpecName "kube-api-access-g6zqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:11:49.232937 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.232916    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-dshm\") pod \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") "
Sep 14 03:11:49.233047 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.232944    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b6ef270-f2f5-48ac-8b8a-8e4297318330-tls-certs\") pod \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\" (UID: \"9b6ef270-f2f5-48ac-8b8a-8e4297318330\") "
Sep 14 03:11:49.233178 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.233165    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:49.233217 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.233181    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6zqh\" (UniqueName: \"kubernetes.io/projected/9b6ef270-f2f5-48ac-8b8a-8e4297318330-kube-api-access-g6zqh\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:49.233217 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.233189    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:49.233217 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.233198    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:49.234981 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.234954    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-dshm" (OuterVolumeSpecName: "dshm") pod "9b6ef270-f2f5-48ac-8b8a-8e4297318330" (UID: "9b6ef270-f2f5-48ac-8b8a-8e4297318330"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:11:49.235195 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.235176    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b6ef270-f2f5-48ac-8b8a-8e4297318330-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9b6ef270-f2f5-48ac-8b8a-8e4297318330" (UID: "9b6ef270-f2f5-48ac-8b8a-8e4297318330"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:11:49.333967 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.333917    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b6ef270-f2f5-48ac-8b8a-8e4297318330-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:49.333967 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.333937    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b6ef270-f2f5-48ac-8b8a-8e4297318330-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:49.429798 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.429768    2574 generic.go:356] "Generic (PLEG): container finished" podID="9b6ef270-f2f5-48ac-8b8a-8e4297318330" containerID="5b33497d5f5d58c81158ca8c6cc50572cb5909b70328502e5de24191175db777" exitCode=0
Sep 14 03:11:49.429929 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.429821    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"
Sep 14 03:11:49.429929 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.429848    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp" event={"ID":"9b6ef270-f2f5-48ac-8b8a-8e4297318330","Type":"ContainerDied","Data":"5b33497d5f5d58c81158ca8c6cc50572cb5909b70328502e5de24191175db777"}
Sep 14 03:11:49.429929 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.429889    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp" event={"ID":"9b6ef270-f2f5-48ac-8b8a-8e4297318330","Type":"ContainerDied","Data":"bf4be3807670f2a21a4075cd195f405c7e3886b820c5adc903e5813da2937381"}
Sep 14 03:11:49.429929 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.429905    2574 scope.go:117] "RemoveContainer" containerID="5b33497d5f5d58c81158ca8c6cc50572cb5909b70328502e5de24191175db777"
Sep 14 03:11:49.438568 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.438549    2574 scope.go:117] "RemoveContainer" containerID="5b33497d5f5d58c81158ca8c6cc50572cb5909b70328502e5de24191175db777"
Sep 14 03:11:49.438846 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:49.438826    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b33497d5f5d58c81158ca8c6cc50572cb5909b70328502e5de24191175db777\": container with ID starting with 5b33497d5f5d58c81158ca8c6cc50572cb5909b70328502e5de24191175db777 not found: ID does not exist" containerID="5b33497d5f5d58c81158ca8c6cc50572cb5909b70328502e5de24191175db777"
Sep 14 03:11:49.438919 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.438858    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b33497d5f5d58c81158ca8c6cc50572cb5909b70328502e5de24191175db777"} err="failed to get container status \"5b33497d5f5d58c81158ca8c6cc50572cb5909b70328502e5de24191175db777\": rpc error: code = NotFound desc = could not find container \"5b33497d5f5d58c81158ca8c6cc50572cb5909b70328502e5de24191175db777\": container with ID starting with 5b33497d5f5d58c81158ca8c6cc50572cb5909b70328502e5de24191175db777 not found: ID does not exist"
Sep 14 03:11:49.456932 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.456906    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"]
Sep 14 03:11:49.459176 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.459153    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/auth-test-model-0c919e-kserve-75bdf8695f-7pkdp"]
Sep 14 03:11:49.599769 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.599680    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb" podUID="080026c0-93d7-4fc0-8807-999827fcd7c7" containerName="main" containerID="cri-o://5df9ed368f22ef41f6edd9010063031e9f679d75003d8bf95b0cbef8ddb835a1" gracePeriod=45
Sep 14 03:11:49.848291 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.848269    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:11:49.937737 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.937652    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-dshm\") pod \"080026c0-93d7-4fc0-8807-999827fcd7c7\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") "
Sep 14 03:11:49.937737 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.937712    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080026c0-93d7-4fc0-8807-999827fcd7c7-tls-certs\") pod \"080026c0-93d7-4fc0-8807-999827fcd7c7\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") "
Sep 14 03:11:49.937737 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.937730    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7p4l\" (UniqueName: \"kubernetes.io/projected/080026c0-93d7-4fc0-8807-999827fcd7c7-kube-api-access-r7p4l\") pod \"080026c0-93d7-4fc0-8807-999827fcd7c7\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") "
Sep 14 03:11:49.938030 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.937748    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-model-cache\") pod \"080026c0-93d7-4fc0-8807-999827fcd7c7\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") "
Sep 14 03:11:49.938030 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.937780    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-tmp-dir\") pod \"080026c0-93d7-4fc0-8807-999827fcd7c7\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") "
Sep 14 03:11:49.938030 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.937821    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-home\") pod \"080026c0-93d7-4fc0-8807-999827fcd7c7\" (UID: \"080026c0-93d7-4fc0-8807-999827fcd7c7\") "
Sep 14 03:11:49.938262 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.938224    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "080026c0-93d7-4fc0-8807-999827fcd7c7" (UID: "080026c0-93d7-4fc0-8807-999827fcd7c7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:11:49.938262 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.938251    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-home" (OuterVolumeSpecName: "home") pod "080026c0-93d7-4fc0-8807-999827fcd7c7" (UID: "080026c0-93d7-4fc0-8807-999827fcd7c7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:11:49.938638 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.938294    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-model-cache" (OuterVolumeSpecName: "model-cache") pod "080026c0-93d7-4fc0-8807-999827fcd7c7" (UID: "080026c0-93d7-4fc0-8807-999827fcd7c7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:11:49.940207 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.940179    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/080026c0-93d7-4fc0-8807-999827fcd7c7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "080026c0-93d7-4fc0-8807-999827fcd7c7" (UID: "080026c0-93d7-4fc0-8807-999827fcd7c7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:11:49.940310 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.940244    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-dshm" (OuterVolumeSpecName: "dshm") pod "080026c0-93d7-4fc0-8807-999827fcd7c7" (UID: "080026c0-93d7-4fc0-8807-999827fcd7c7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:11:49.940373 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:49.940348    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/080026c0-93d7-4fc0-8807-999827fcd7c7-kube-api-access-r7p4l" (OuterVolumeSpecName: "kube-api-access-r7p4l") pod "080026c0-93d7-4fc0-8807-999827fcd7c7" (UID: "080026c0-93d7-4fc0-8807-999827fcd7c7"). InnerVolumeSpecName "kube-api-access-r7p4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:11:50.038570 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.038533    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:50.038570 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.038565    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/080026c0-93d7-4fc0-8807-999827fcd7c7-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:50.038570 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.038575    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7p4l\" (UniqueName: \"kubernetes.io/projected/080026c0-93d7-4fc0-8807-999827fcd7c7-kube-api-access-r7p4l\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:50.038816 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.038584    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:50.038816 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.038594    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:50.038816 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.038602    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/080026c0-93d7-4fc0-8807-999827fcd7c7-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:11:50.218259 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.218189    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b6ef270-f2f5-48ac-8b8a-8e4297318330" path="/var/lib/kubelet/pods/9b6ef270-f2f5-48ac-8b8a-8e4297318330/volumes"
Sep 14 03:11:50.437084 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.437052    2574 generic.go:356] "Generic (PLEG): container finished" podID="080026c0-93d7-4fc0-8807-999827fcd7c7" containerID="5df9ed368f22ef41f6edd9010063031e9f679d75003d8bf95b0cbef8ddb835a1" exitCode=0
Sep 14 03:11:50.437260 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.437110    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"
Sep 14 03:11:50.437260 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.437142    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb" event={"ID":"080026c0-93d7-4fc0-8807-999827fcd7c7","Type":"ContainerDied","Data":"5df9ed368f22ef41f6edd9010063031e9f679d75003d8bf95b0cbef8ddb835a1"}
Sep 14 03:11:50.437260 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.437189    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb" event={"ID":"080026c0-93d7-4fc0-8807-999827fcd7c7","Type":"ContainerDied","Data":"fb87ddcf933a21d0773795fb018dc8bab966fac71003c1d1b59f787664b6eb0f"}
Sep 14 03:11:50.437260 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.437213    2574 scope.go:117] "RemoveContainer" containerID="5df9ed368f22ef41f6edd9010063031e9f679d75003d8bf95b0cbef8ddb835a1"
Sep 14 03:11:50.447270 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.447253    2574 scope.go:117] "RemoveContainer" containerID="5df9ed368f22ef41f6edd9010063031e9f679d75003d8bf95b0cbef8ddb835a1"
Sep 14 03:11:50.447585 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:11:50.447566    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5df9ed368f22ef41f6edd9010063031e9f679d75003d8bf95b0cbef8ddb835a1\": container with ID starting with 5df9ed368f22ef41f6edd9010063031e9f679d75003d8bf95b0cbef8ddb835a1 not found: ID does not exist" containerID="5df9ed368f22ef41f6edd9010063031e9f679d75003d8bf95b0cbef8ddb835a1"
Sep 14 03:11:50.447665 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.447597    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5df9ed368f22ef41f6edd9010063031e9f679d75003d8bf95b0cbef8ddb835a1"} err="failed to get container status \"5df9ed368f22ef41f6edd9010063031e9f679d75003d8bf95b0cbef8ddb835a1\": rpc error: code = NotFound desc = could not find container \"5df9ed368f22ef41f6edd9010063031e9f679d75003d8bf95b0cbef8ddb835a1\": container with ID starting with 5df9ed368f22ef41f6edd9010063031e9f679d75003d8bf95b0cbef8ddb835a1 not found: ID does not exist"
Sep 14 03:11:50.454882 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.454858    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"]
Sep 14 03:11:50.459415 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:50.459388    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/auth-test-model-f83c87-kserve-69758b87dc-qblsb"]
Sep 14 03:11:52.216882 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:52.216853    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="080026c0-93d7-4fc0-8807-999827fcd7c7" path="/var/lib/kubelet/pods/080026c0-93d7-4fc0-8807-999827fcd7c7/volumes"
Sep 14 03:11:52.456556 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:52.456455    2574 generic.go:356] "Generic (PLEG): container finished" podID="b3bf449c-92d9-42a0-a589-85fb9862e51a" containerID="bb0037605705cd19aa640c220f1001e0e2caea74aa9f4a59586b741bb3fe4e5b" exitCode=0
Sep 14 03:11:52.456679 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:52.456552    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz" event={"ID":"b3bf449c-92d9-42a0-a589-85fb9862e51a","Type":"ContainerDied","Data":"bb0037605705cd19aa640c220f1001e0e2caea74aa9f4a59586b741bb3fe4e5b"}
Sep 14 03:11:52.456967 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:52.456953    2574 scope.go:117] "RemoveContainer" containerID="bb0037605705cd19aa640c220f1001e0e2caea74aa9f4a59586b741bb3fe4e5b"
Sep 14 03:11:52.458103 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:52.458074    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:11:52.458199 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:52.458154    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:11:52.458199 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:52.458189    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:11:53.463408 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:53.463377    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz" event={"ID":"b3bf449c-92d9-42a0-a589-85fb9862e51a","Type":"ContainerStarted","Data":"f8035f755b1818c652ccf26ddccfca3363cb87dc09383a6485bfe24235897763"}
Sep 14 03:11:54.066135 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:54.066053    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:54.066337 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:54.066297    2574 patch_prober.go:28] interesting pod/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 03:11:54.066420 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:54.066343    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz" podUID="b3bf449c-92d9-42a0-a589-85fb9862e51a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 03:11:54.263179 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:54.263149    2574 scope.go:117] "RemoveContainer" containerID="2a7a4d49158e8b5258009fa8e56acf314e1d094b9e897ca7660bb5dffd81a10f"
Sep 14 03:11:55.065336 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:55.065295    2574 patch_prober.go:28] interesting pod/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 03:11:55.065801 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:55.065346    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz" podUID="b3bf449c-92d9-42a0-a589-85fb9862e51a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 03:11:56.066195 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:56.066159    2574 patch_prober.go:28] interesting pod/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 03:11:56.066843 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:56.066809    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz" podUID="b3bf449c-92d9-42a0-a589-85fb9862e51a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 03:11:57.066166 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:57.066122    2574 patch_prober.go:28] interesting pod/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 03:11:57.066368 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:57.066189    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz" podUID="b3bf449c-92d9-42a0-a589-85fb9862e51a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 03:11:58.065300 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:58.065266    2574 patch_prober.go:28] interesting pod/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 03:11:58.065478 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:58.065322    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz" podUID="b3bf449c-92d9-42a0-a589-85fb9862e51a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 03:11:59.069253 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:59.069226    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:59.069672 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:59.069554    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:11:59.070207 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:11:59.070185    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:12:10.985988 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:10.985959    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"]
Sep 14 03:12:10.986335 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:10.986289    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="080026c0-93d7-4fc0-8807-999827fcd7c7" containerName="main"
Sep 14 03:12:10.986335 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:10.986299    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="080026c0-93d7-4fc0-8807-999827fcd7c7" containerName="main"
Sep 14 03:12:10.986335 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:10.986315    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6ef270-f2f5-48ac-8b8a-8e4297318330" containerName="main"
Sep 14 03:12:10.986335 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:10.986320    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6ef270-f2f5-48ac-8b8a-8e4297318330" containerName="main"
Sep 14 03:12:10.986335 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:10.986334    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce90edac-2863-437f-a37f-d9b84d939c0c" containerName="test-curl-8996-5eebf6"
Sep 14 03:12:10.986495 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:10.986340    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce90edac-2863-437f-a37f-d9b84d939c0c" containerName="test-curl-8996-5eebf6"
Sep 14 03:12:10.986495 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:10.986400    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce90edac-2863-437f-a37f-d9b84d939c0c" containerName="test-curl-8996-5eebf6"
Sep 14 03:12:10.986495 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:10.986410    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6ef270-f2f5-48ac-8b8a-8e4297318330" containerName="main"
Sep 14 03:12:10.986495 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:10.986417    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="080026c0-93d7-4fc0-8807-999827fcd7c7" containerName="main"
Sep 14 03:12:10.990713 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:10.990696    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:10.994480 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:10.994458    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-0c919e\"/\"sub-test-model-0c919e-kserve-self-signed-certs\""
Sep 14 03:12:10.995592 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:10.995573    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-0c919e\"/\"openshift-service-ca.crt\""
Sep 14 03:12:10.995592 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:10.995584    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-0c919e\"/\"kube-root-ca.crt\""
Sep 14 03:12:10.995765 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:10.995646    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-0c919e\"/\"default-dockercfg-llp22\""
Sep 14 03:12:11.007493 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.007470    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"]
Sep 14 03:12:11.022077 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.022055    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-model-cache\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.022162 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.022095    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-dshm\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.022228 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.022202    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-home\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.022334 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.022243    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/604aed98-fc2c-4037-a9d5-3b383336e0f8-tls-certs\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.022334 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.022275    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-tmp-dir\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.022334 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.022291    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wkz9\" (UniqueName: \"kubernetes.io/projected/604aed98-fc2c-4037-a9d5-3b383336e0f8-kube-api-access-5wkz9\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.123438 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.123411    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-dshm\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.123567 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.123469    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-home\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.123567 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.123494    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/604aed98-fc2c-4037-a9d5-3b383336e0f8-tls-certs\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.123567 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.123545    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-tmp-dir\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.123567 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.123566    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wkz9\" (UniqueName: \"kubernetes.io/projected/604aed98-fc2c-4037-a9d5-3b383336e0f8-kube-api-access-5wkz9\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.123772 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.123605    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-model-cache\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.123935 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.123911    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-home\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.124003 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.123942    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-tmp-dir\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.124003 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.123963    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-model-cache\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.125945 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.125925    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-dshm\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.126117 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.126100    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/604aed98-fc2c-4037-a9d5-3b383336e0f8-tls-certs\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.133172 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.133153    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wkz9\" (UniqueName: \"kubernetes.io/projected/604aed98-fc2c-4037-a9d5-3b383336e0f8-kube-api-access-5wkz9\") pod \"sub-test-model-0c919e-kserve-55f8f4669-jrwdh\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.299700 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.299604    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:11.574222 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:11.574145    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"]
Sep 14 03:12:11.577166 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:12:11.577138    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod604aed98_fc2c_4037_a9d5_3b383336e0f8.slice/crio-8fc17c6b1a724a90adef2f03d94bfb27b9700d64afcac2252340e2bd6cffc2c4 WatchSource:0}: Error finding container 8fc17c6b1a724a90adef2f03d94bfb27b9700d64afcac2252340e2bd6cffc2c4: Status 404 returned error can't find the container with id 8fc17c6b1a724a90adef2f03d94bfb27b9700d64afcac2252340e2bd6cffc2c4
Sep 14 03:12:12.576371 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:12.576337    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh" event={"ID":"604aed98-fc2c-4037-a9d5-3b383336e0f8","Type":"ContainerStarted","Data":"89eb282be071c8fc6b193656e78e02dc8dad0df638b51ed985bdef7dac1ec478"}
Sep 14 03:12:12.576796 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:12.576382    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh" event={"ID":"604aed98-fc2c-4037-a9d5-3b383336e0f8","Type":"ContainerStarted","Data":"8fc17c6b1a724a90adef2f03d94bfb27b9700d64afcac2252340e2bd6cffc2c4"}
Sep 14 03:12:12.610222 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:12.610175    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh" podStartSLOduration=2.610159789 podStartE2EDuration="2.610159789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:12:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:12:12.60900093 +0000 UTC m=+1278.922024748" watchObservedRunningTime="2026-09-14 03:12:12.610159789 +0000 UTC m=+1278.923183608"
Sep 14 03:12:15.152713 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:15.152679    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:12:20.993659 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:20.993618    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"]
Sep 14 03:12:20.997086 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:20.997069    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.005386 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.005363    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-d6ac6dc8\"/\"kube-root-ca.crt\""
Sep 14 03:12:21.005386 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.005375    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-d6ac6dc8\"/\"openshift-service-ca.crt\""
Sep 14 03:12:21.005571 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.005370    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-d6ac6dc8\"/\"e2e-lifecycle-model-d6ac6dc8-kserve-self-signed-certs\""
Sep 14 03:12:21.005571 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.005400    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-d6ac6dc8\"/\"default-dockercfg-hm4mp\""
Sep 14 03:12:21.012133 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.012113    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"]
Sep 14 03:12:21.111303 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.111279    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-model-cache\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.111462 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.111329    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhcsj\" (UniqueName: \"kubernetes.io/projected/9bef1150-42c0-465e-a2a2-480c69e7e9e5-kube-api-access-fhcsj\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.111462 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.111427    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-home\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.111607 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.111481    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-dshm\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.111607 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.111507    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-tmp-dir\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.111607 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.111585    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bef1150-42c0-465e-a2a2-480c69e7e9e5-tls-certs\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.212679 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.212653    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-tmp-dir\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.212798 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.212700    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bef1150-42c0-465e-a2a2-480c69e7e9e5-tls-certs\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.212798 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.212736    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-model-cache\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.212880 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.212821    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhcsj\" (UniqueName: \"kubernetes.io/projected/9bef1150-42c0-465e-a2a2-480c69e7e9e5-kube-api-access-fhcsj\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.212918 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.212880    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-home\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.212953 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.212922    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-dshm\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.213141 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.213120    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-tmp-dir\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.213184 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.213152    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-model-cache\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.213276 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.213256    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-home\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.215294 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.215275    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-dshm\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.215433 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.215412    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bef1150-42c0-465e-a2a2-480c69e7e9e5-tls-certs\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.222226 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.222208    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhcsj\" (UniqueName: \"kubernetes.io/projected/9bef1150-42c0-465e-a2a2-480c69e7e9e5-kube-api-access-fhcsj\") pod \"e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") " pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.300242 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.300152    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:21.300436 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.300309    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:21.306093 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.306072    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:21.313144 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.313122    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:21.580026 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.580002    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"]
Sep 14 03:12:21.581667 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:12:21.581640    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9bef1150_42c0_465e_a2a2_480c69e7e9e5.slice/crio-9696b8629c8deb0395cb191cd6173e4a32963529fd6b5e38107b4f8e30a101cb WatchSource:0}: Error finding container 9696b8629c8deb0395cb191cd6173e4a32963529fd6b5e38107b4f8e30a101cb: Status 404 returned error can't find the container with id 9696b8629c8deb0395cb191cd6173e4a32963529fd6b5e38107b4f8e30a101cb
Sep 14 03:12:21.623572 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.623546    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d" event={"ID":"9bef1150-42c0-465e-a2a2-480c69e7e9e5","Type":"ContainerStarted","Data":"9696b8629c8deb0395cb191cd6173e4a32963529fd6b5e38107b4f8e30a101cb"}
Sep 14 03:12:21.635577 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:21.635552    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:12:22.630339 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:22.630294    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d" event={"ID":"9bef1150-42c0-465e-a2a2-480c69e7e9e5","Type":"ContainerStarted","Data":"597461b9cc3b3aaaab202521d4d05ab04ebf182840630264d7e542f43610cf27"}
Sep 14 03:12:22.657300 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:22.657252    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d" podStartSLOduration=2.657236301 podStartE2EDuration="2.657236301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:12:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:12:22.654322032 +0000 UTC m=+1288.967345850" watchObservedRunningTime="2026-09-14 03:12:22.657236301 +0000 UTC m=+1288.970260120"
Sep 14 03:12:27.142437 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:27.142402    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:12:31.306405 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:31.306372    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:31.306816 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:31.306416    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:31.319056 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:31.319035    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:31.686279 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:31.686205    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:12:50.193403 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.193366    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"]
Sep 14 03:12:50.197089 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.197066    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.199774 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.199749    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-f83c87\"/\"openshift-service-ca.crt\""
Sep 14 03:12:50.200068 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.200042    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-f83c87\"/\"kube-root-ca.crt\""
Sep 14 03:12:50.201195 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.201177    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-f83c87\"/\"default-dockercfg-j84w5\""
Sep 14 03:12:50.201290 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.201213    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-f83c87\"/\"sub-test-model-f83c87-kserve-self-signed-certs\""
Sep 14 03:12:50.208464 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.208440    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"]
Sep 14 03:12:50.267234 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.267208    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-model-cache\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.267333 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.267251    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-home\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.267333 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.267320    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-tmp-dir\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.267419 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.267371    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnjvq\" (UniqueName: \"kubernetes.io/projected/6dbe09a5-e402-43e4-b84b-788354eb3d37-kube-api-access-rnjvq\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.267419 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.267402    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-dshm\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.267494 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.267432    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dbe09a5-e402-43e4-b84b-788354eb3d37-tls-certs\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.368314 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.368280    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnjvq\" (UniqueName: \"kubernetes.io/projected/6dbe09a5-e402-43e4-b84b-788354eb3d37-kube-api-access-rnjvq\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.368482 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.368329    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-dshm\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.368482 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.368351    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dbe09a5-e402-43e4-b84b-788354eb3d37-tls-certs\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.368620 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.368474    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-model-cache\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.368620 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.368583    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-home\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.368730 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.368677    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-tmp-dir\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.368879 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.368849    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-model-cache\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.369019 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.368942    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-home\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.369019 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.369000    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-tmp-dir\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.370866 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.370847    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-dshm\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.370979 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.370962    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dbe09a5-e402-43e4-b84b-788354eb3d37-tls-certs\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.377564 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.377543    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnjvq\" (UniqueName: \"kubernetes.io/projected/6dbe09a5-e402-43e4-b84b-788354eb3d37-kube-api-access-rnjvq\") pod \"sub-test-model-f83c87-kserve-559b5847f7-6rhnr\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.508437 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.508406    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:12:50.781613 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:50.780643    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"]
Sep 14 03:12:50.783950 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:12:50.783910    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6dbe09a5_e402_43e4_b84b_788354eb3d37.slice/crio-fe3606eaea6156ab2bf7044457a89cc6a5310d6bde4350c34d2e8902ba5c4b28 WatchSource:0}: Error finding container fe3606eaea6156ab2bf7044457a89cc6a5310d6bde4350c34d2e8902ba5c4b28: Status 404 returned error can't find the container with id fe3606eaea6156ab2bf7044457a89cc6a5310d6bde4350c34d2e8902ba5c4b28
Sep 14 03:12:51.788317 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:51.788281    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr" event={"ID":"6dbe09a5-e402-43e4-b84b-788354eb3d37","Type":"ContainerStarted","Data":"5fda7d3acee8e9079d2cac9c6726e70506d1f086c36db302ee4e4309ce408e7d"}
Sep 14 03:12:51.788317 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:51.788319    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr" event={"ID":"6dbe09a5-e402-43e4-b84b-788354eb3d37","Type":"ContainerStarted","Data":"fe3606eaea6156ab2bf7044457a89cc6a5310d6bde4350c34d2e8902ba5c4b28"}
Sep 14 03:12:51.807174 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:51.807129    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr" podStartSLOduration=1.807112563 podStartE2EDuration="1.807112563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:12:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:12:51.806777923 +0000 UTC m=+1318.119801742" watchObservedRunningTime="2026-09-14 03:12:51.807112563 +0000 UTC m=+1318.120136432"
Sep 14 03:12:54.321979 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:54.321949    2574 scope.go:117] "RemoveContainer" containerID="2ee7b5fd5e66cc5624a9be6b63d8b1bfafc5556915f9ab9f14402e715747b7d0"
Sep 14 03:12:55.438588 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:12:55.438556    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:13:00.509505 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:00.509468    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:13:00.509922 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:00.509539    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:13:00.521335 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:00.521310    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:13:00.845913 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:00.845833    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:13:06.426587 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:06.426558    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr"]
Sep 14 03:13:06.431383 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:06.431358    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr"
Sep 14 03:13:06.434103 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:06.434081    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:13:06.434226 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:06.434124    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4hf5p\""
Sep 14 03:13:06.439200 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:06.439170    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr"]
Sep 14 03:13:06.504256 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:06.504222    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e92b675-1924-42d1-a79d-4fe468a74cea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr\" (UID: \"2e92b675-1924-42d1-a79d-4fe468a74cea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr"
Sep 14 03:13:06.605190 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:06.605160    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e92b675-1924-42d1-a79d-4fe468a74cea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr\" (UID: \"2e92b675-1924-42d1-a79d-4fe468a74cea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr"
Sep 14 03:13:06.605777 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:06.605757    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e92b675-1924-42d1-a79d-4fe468a74cea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr\" (UID: \"2e92b675-1924-42d1-a79d-4fe468a74cea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr"
Sep 14 03:13:06.743451 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:06.743425    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr"
Sep 14 03:13:07.036439 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:07.036390    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr"]
Sep 14 03:13:07.038695 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:13:07.038661    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e92b675_1924_42d1_a79d_4fe468a74cea.slice/crio-280616f2e2df0babae000485d71d38288ade313f08bc20c932441f7f599e7a7e WatchSource:0}: Error finding container 280616f2e2df0babae000485d71d38288ade313f08bc20c932441f7f599e7a7e: Status 404 returned error can't find the container with id 280616f2e2df0babae000485d71d38288ade313f08bc20c932441f7f599e7a7e
Sep 14 03:13:07.874095 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:07.874061    2574 generic.go:356] "Generic (PLEG): container finished" podID="2e92b675-1924-42d1-a79d-4fe468a74cea" containerID="7abfe487d89677439d3751f3f8f55526e1ebe1c301af57aafefeb661365af44e" exitCode=0
Sep 14 03:13:07.874411 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:07.874112    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr" event={"ID":"2e92b675-1924-42d1-a79d-4fe468a74cea","Type":"ContainerDied","Data":"7abfe487d89677439d3751f3f8f55526e1ebe1c301af57aafefeb661365af44e"}
Sep 14 03:13:07.874411 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:07.874168    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr" event={"ID":"2e92b675-1924-42d1-a79d-4fe468a74cea","Type":"ContainerStarted","Data":"280616f2e2df0babae000485d71d38288ade313f08bc20c932441f7f599e7a7e"}
Sep 14 03:13:08.747198 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:08.747166    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:13:09.029813 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:09.029789    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr"
Sep 14 03:13:09.127772 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:09.127751    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e92b675-1924-42d1-a79d-4fe468a74cea-maas-api-service-ca\") pod \"2e92b675-1924-42d1-a79d-4fe468a74cea\" (UID: \"2e92b675-1924-42d1-a79d-4fe468a74cea\") "
Sep 14 03:13:09.128131 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:09.128110    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e92b675-1924-42d1-a79d-4fe468a74cea-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2e92b675-1924-42d1-a79d-4fe468a74cea" (UID: "2e92b675-1924-42d1-a79d-4fe468a74cea"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:13:09.228772 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:09.228750    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e92b675-1924-42d1-a79d-4fe468a74cea-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:09.885445 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:09.885417    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr"
Sep 14 03:13:09.885633 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:09.885408    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr" event={"ID":"2e92b675-1924-42d1-a79d-4fe468a74cea","Type":"ContainerDied","Data":"280616f2e2df0babae000485d71d38288ade313f08bc20c932441f7f599e7a7e"}
Sep 14 03:13:09.885633 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:09.885592    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="280616f2e2df0babae000485d71d38288ade313f08bc20c932441f7f599e7a7e"
Sep 14 03:13:22.225164 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:22.225095    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr"]
Sep 14 03:13:22.225164 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:22.225130    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d6ac6dc8-97xnr"]
Sep 14 03:13:22.397978 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:22.397942    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8996-cc4402"]
Sep 14 03:13:22.398311 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:22.398299    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e92b675-1924-42d1-a79d-4fe468a74cea" containerName="revoke-keys"
Sep 14 03:13:22.398356 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:22.398313    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e92b675-1924-42d1-a79d-4fe468a74cea" containerName="revoke-keys"
Sep 14 03:13:22.398396 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:22.398371    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e92b675-1924-42d1-a79d-4fe468a74cea" containerName="revoke-keys"
Sep 14 03:13:22.401146 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:22.401130    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8996-cc4402"
Sep 14 03:13:22.404001 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:22.403982    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4ff8v\""
Sep 14 03:13:22.408602 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:22.408579    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8996-cc4402"]
Sep 14 03:13:22.545250 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:22.545224    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgtkf\" (UniqueName: \"kubernetes.io/projected/e066738f-014e-4ba8-bc21-f0cf244e4dfd-kube-api-access-mgtkf\") pod \"test-curl-8996-cc4402\" (UID: \"e066738f-014e-4ba8-bc21-f0cf244e4dfd\") " pod="opendatahub/test-curl-8996-cc4402"
Sep 14 03:13:22.646623 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:22.646599    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgtkf\" (UniqueName: \"kubernetes.io/projected/e066738f-014e-4ba8-bc21-f0cf244e4dfd-kube-api-access-mgtkf\") pod \"test-curl-8996-cc4402\" (UID: \"e066738f-014e-4ba8-bc21-f0cf244e4dfd\") " pod="opendatahub/test-curl-8996-cc4402"
Sep 14 03:13:22.664304 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:22.664279    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgtkf\" (UniqueName: \"kubernetes.io/projected/e066738f-014e-4ba8-bc21-f0cf244e4dfd-kube-api-access-mgtkf\") pod \"test-curl-8996-cc4402\" (UID: \"e066738f-014e-4ba8-bc21-f0cf244e4dfd\") " pod="opendatahub/test-curl-8996-cc4402"
Sep 14 03:13:22.711068 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:22.711039    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8996-cc4402"
Sep 14 03:13:22.981193 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:22.981164    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8996-cc4402"]
Sep 14 03:13:22.982861 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:13:22.982830    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode066738f_014e_4ba8_bc21_f0cf244e4dfd.slice/crio-0e8492b6291f698bfaf788af1d1655a788b13fa4acba3c5844990c644b4853aa WatchSource:0}: Error finding container 0e8492b6291f698bfaf788af1d1655a788b13fa4acba3c5844990c644b4853aa: Status 404 returned error can't find the container with id 0e8492b6291f698bfaf788af1d1655a788b13fa4acba3c5844990c644b4853aa
Sep 14 03:13:23.963572 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:23.963536    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8996-cc4402" event={"ID":"e066738f-014e-4ba8-bc21-f0cf244e4dfd","Type":"ContainerStarted","Data":"f1944e1d29fb39c7619e55399e6c9d4e03ec15a72ca29088def358d94a97c342"}
Sep 14 03:13:23.963572 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:23.963581    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8996-cc4402" event={"ID":"e066738f-014e-4ba8-bc21-f0cf244e4dfd","Type":"ContainerStarted","Data":"0e8492b6291f698bfaf788af1d1655a788b13fa4acba3c5844990c644b4853aa"}
Sep 14 03:13:23.983838 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:23.983790    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8996-cc4402" podStartSLOduration=1.983775404 podStartE2EDuration="1.983775404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:13:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:13:23.982160069 +0000 UTC m=+1350.295183888" watchObservedRunningTime="2026-09-14 03:13:23.983775404 +0000 UTC m=+1350.296799223"
Sep 14 03:13:24.216531 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:24.216445    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e92b675-1924-42d1-a79d-4fe468a74cea" path="/var/lib/kubelet/pods/2e92b675-1924-42d1-a79d-4fe468a74cea/volumes"
Sep 14 03:13:24.574821 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:24.574782    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8996-cc4402"]
Sep 14 03:13:24.577835 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:24.577814    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8996-cc4402"]
Sep 14 03:13:24.687546 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:24.687506    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8996-c5633a"]
Sep 14 03:13:24.687895 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:24.687883    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e066738f-014e-4ba8-bc21-f0cf244e4dfd" containerName="test-curl-8996-cc4402"
Sep 14 03:13:24.687947 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:24.687897    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e066738f-014e-4ba8-bc21-f0cf244e4dfd" containerName="test-curl-8996-cc4402"
Sep 14 03:13:24.687990 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:24.687967    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e066738f-014e-4ba8-bc21-f0cf244e4dfd" containerName="test-curl-8996-cc4402"
Sep 14 03:13:24.690959 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:24.690940    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8996-c5633a"
Sep 14 03:13:24.697840 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:24.697816    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8996-c5633a"]
Sep 14 03:13:24.867852 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:24.867783    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szkhx\" (UniqueName: \"kubernetes.io/projected/89e994ea-c41a-4752-b6b3-00c18a4da521-kube-api-access-szkhx\") pod \"test-curl-8996-c5633a\" (UID: \"89e994ea-c41a-4752-b6b3-00c18a4da521\") " pod="opendatahub/test-curl-8996-c5633a"
Sep 14 03:13:24.968870 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:24.968834    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szkhx\" (UniqueName: \"kubernetes.io/projected/89e994ea-c41a-4752-b6b3-00c18a4da521-kube-api-access-szkhx\") pod \"test-curl-8996-c5633a\" (UID: \"89e994ea-c41a-4752-b6b3-00c18a4da521\") " pod="opendatahub/test-curl-8996-c5633a"
Sep 14 03:13:24.969402 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:13:24.969383    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8996-cc4402" container="test-curl-8996-cc4402"
Sep 14 03:13:24.978645 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:24.978502    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szkhx\" (UniqueName: \"kubernetes.io/projected/89e994ea-c41a-4752-b6b3-00c18a4da521-kube-api-access-szkhx\") pod \"test-curl-8996-c5633a\" (UID: \"89e994ea-c41a-4752-b6b3-00c18a4da521\") " pod="opendatahub/test-curl-8996-c5633a"
Sep 14 03:13:25.001266 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:25.001241    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8996-c5633a"
Sep 14 03:13:25.273224 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:25.273198    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8996-c5633a"]
Sep 14 03:13:25.276485 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:13:25.276454    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89e994ea_c41a_4752_b6b3_00c18a4da521.slice/crio-2cb79e86361abf9a90e4f8539b0ad2663000af8ce96f46ca5a8dd40fdb27055a WatchSource:0}: Error finding container 2cb79e86361abf9a90e4f8539b0ad2663000af8ce96f46ca5a8dd40fdb27055a: Status 404 returned error can't find the container with id 2cb79e86361abf9a90e4f8539b0ad2663000af8ce96f46ca5a8dd40fdb27055a
Sep 14 03:13:25.976457 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:25.976418    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8996-c5633a" event={"ID":"89e994ea-c41a-4752-b6b3-00c18a4da521","Type":"ContainerStarted","Data":"716dbcde722f4a3e63e12e1bef0fe72db44c4328681139ffca279e323e7ffbf9"}
Sep 14 03:13:25.976910 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:25.976468    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8996-c5633a" event={"ID":"89e994ea-c41a-4752-b6b3-00c18a4da521","Type":"ContainerStarted","Data":"2cb79e86361abf9a90e4f8539b0ad2663000af8ce96f46ca5a8dd40fdb27055a"}
Sep 14 03:13:25.976910 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:25.976585    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8996-cc4402" podUID="e066738f-014e-4ba8-bc21-f0cf244e4dfd" containerName="test-curl-8996-cc4402" containerID="cri-o://f1944e1d29fb39c7619e55399e6c9d4e03ec15a72ca29088def358d94a97c342" gracePeriod=2
Sep 14 03:13:25.992204 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:25.992162    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8996-c5633a" podStartSLOduration=1.992149807 podStartE2EDuration="1.992149807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:13:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:13:25.99091238 +0000 UTC m=+1352.303936198" watchObservedRunningTime="2026-09-14 03:13:25.992149807 +0000 UTC m=+1352.305173625"
Sep 14 03:13:25.992973 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:13:25.992945    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8996-cc4402\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" podUID="e066738f-014e-4ba8-bc21-f0cf244e4dfd" pod="opendatahub/test-curl-8996-cc4402"
Sep 14 03:13:26.588800 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:26.588766    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8996-c5633a"]
Sep 14 03:13:26.593200 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:26.593158    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8996-c5633a"]
Sep 14 03:13:26.981700 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:13:26.981666    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8996-c5633a" container="test-curl-8996-c5633a"
Sep 14 03:13:27.987144 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:27.987112    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8996-c5633a" podUID="89e994ea-c41a-4752-b6b3-00c18a4da521" containerName="test-curl-8996-c5633a" containerID="cri-o://716dbcde722f4a3e63e12e1bef0fe72db44c4328681139ffca279e323e7ffbf9" gracePeriod=2
Sep 14 03:13:27.989654 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:13:27.989609    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8996-c5633a\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" podUID="89e994ea-c41a-4752-b6b3-00c18a4da521" pod="opendatahub/test-curl-8996-c5633a"
Sep 14 03:13:28.147757 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:28.147734    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8996-cc4402"
Sep 14 03:13:28.150462 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:13:28.150436    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8996-c5633a\" is forbidden: User \"system:node:ip-10-0-133-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-188.ec2.internal' and this object" podUID="89e994ea-c41a-4752-b6b3-00c18a4da521" pod="opendatahub/test-curl-8996-c5633a"
Sep 14 03:13:28.300720 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:28.300655    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgtkf\" (UniqueName: \"kubernetes.io/projected/e066738f-014e-4ba8-bc21-f0cf244e4dfd-kube-api-access-mgtkf\") pod \"e066738f-014e-4ba8-bc21-f0cf244e4dfd\" (UID: \"e066738f-014e-4ba8-bc21-f0cf244e4dfd\") "
Sep 14 03:13:28.302759 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:28.302732    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e066738f-014e-4ba8-bc21-f0cf244e4dfd-kube-api-access-mgtkf" (OuterVolumeSpecName: "kube-api-access-mgtkf") pod "e066738f-014e-4ba8-bc21-f0cf244e4dfd" (UID: "e066738f-014e-4ba8-bc21-f0cf244e4dfd"). InnerVolumeSpecName "kube-api-access-mgtkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:13:28.402108 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:28.402081    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgtkf\" (UniqueName: \"kubernetes.io/projected/e066738f-014e-4ba8-bc21-f0cf244e4dfd-kube-api-access-mgtkf\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:28.599210 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:28.599135    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"]
Sep 14 03:13:28.993375 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:28.993344    2574 generic.go:356] "Generic (PLEG): container finished" podID="e066738f-014e-4ba8-bc21-f0cf244e4dfd" containerID="f1944e1d29fb39c7619e55399e6c9d4e03ec15a72ca29088def358d94a97c342" exitCode=137
Sep 14 03:13:28.993795 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:28.993384    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8996-cc4402"
Sep 14 03:13:28.993795 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:28.993443    2574 scope.go:117] "RemoveContainer" containerID="f1944e1d29fb39c7619e55399e6c9d4e03ec15a72ca29088def358d94a97c342"
Sep 14 03:13:29.002644 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:29.002627    2574 scope.go:117] "RemoveContainer" containerID="f1944e1d29fb39c7619e55399e6c9d4e03ec15a72ca29088def358d94a97c342"
Sep 14 03:13:29.002923 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:13:29.002904    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1944e1d29fb39c7619e55399e6c9d4e03ec15a72ca29088def358d94a97c342\": container with ID starting with f1944e1d29fb39c7619e55399e6c9d4e03ec15a72ca29088def358d94a97c342 not found: ID does not exist" containerID="f1944e1d29fb39c7619e55399e6c9d4e03ec15a72ca29088def358d94a97c342"
Sep 14 03:13:29.002975 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:29.002933    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1944e1d29fb39c7619e55399e6c9d4e03ec15a72ca29088def358d94a97c342"} err="failed to get container status \"f1944e1d29fb39c7619e55399e6c9d4e03ec15a72ca29088def358d94a97c342\": rpc error: code = NotFound desc = could not find container \"f1944e1d29fb39c7619e55399e6c9d4e03ec15a72ca29088def358d94a97c342\": container with ID starting with f1944e1d29fb39c7619e55399e6c9d4e03ec15a72ca29088def358d94a97c342 not found: ID does not exist"
Sep 14 03:13:29.497177 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:29.497147    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"]
Sep 14 03:13:30.164281 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:30.164255    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8996-c5633a"
Sep 14 03:13:30.216094 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:30.216071    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e066738f-014e-4ba8-bc21-f0cf244e4dfd" path="/var/lib/kubelet/pods/e066738f-014e-4ba8-bc21-f0cf244e4dfd/volumes"
Sep 14 03:13:30.317132 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:30.317083    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szkhx\" (UniqueName: \"kubernetes.io/projected/89e994ea-c41a-4752-b6b3-00c18a4da521-kube-api-access-szkhx\") pod \"89e994ea-c41a-4752-b6b3-00c18a4da521\" (UID: \"89e994ea-c41a-4752-b6b3-00c18a4da521\") "
Sep 14 03:13:30.319176 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:30.319151    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89e994ea-c41a-4752-b6b3-00c18a4da521-kube-api-access-szkhx" (OuterVolumeSpecName: "kube-api-access-szkhx") pod "89e994ea-c41a-4752-b6b3-00c18a4da521" (UID: "89e994ea-c41a-4752-b6b3-00c18a4da521"). InnerVolumeSpecName "kube-api-access-szkhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:13:30.347917 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:30.347894    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"]
Sep 14 03:13:30.418799 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:30.418770    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szkhx\" (UniqueName: \"kubernetes.io/projected/89e994ea-c41a-4752-b6b3-00c18a4da521-kube-api-access-szkhx\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:31.007862 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:31.007824    2574 generic.go:356] "Generic (PLEG): container finished" podID="89e994ea-c41a-4752-b6b3-00c18a4da521" containerID="716dbcde722f4a3e63e12e1bef0fe72db44c4328681139ffca279e323e7ffbf9" exitCode=137
Sep 14 03:13:31.008048 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:31.007875    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8996-c5633a"
Sep 14 03:13:31.008048 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:31.007922    2574 scope.go:117] "RemoveContainer" containerID="716dbcde722f4a3e63e12e1bef0fe72db44c4328681139ffca279e323e7ffbf9"
Sep 14 03:13:31.016883 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:31.016864    2574 scope.go:117] "RemoveContainer" containerID="716dbcde722f4a3e63e12e1bef0fe72db44c4328681139ffca279e323e7ffbf9"
Sep 14 03:13:31.017157 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:13:31.017139    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"716dbcde722f4a3e63e12e1bef0fe72db44c4328681139ffca279e323e7ffbf9\": container with ID starting with 716dbcde722f4a3e63e12e1bef0fe72db44c4328681139ffca279e323e7ffbf9 not found: ID does not exist" containerID="716dbcde722f4a3e63e12e1bef0fe72db44c4328681139ffca279e323e7ffbf9"
Sep 14 03:13:31.017212 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:31.017170    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"716dbcde722f4a3e63e12e1bef0fe72db44c4328681139ffca279e323e7ffbf9"} err="failed to get container status \"716dbcde722f4a3e63e12e1bef0fe72db44c4328681139ffca279e323e7ffbf9\": rpc error: code = NotFound desc = could not find container \"716dbcde722f4a3e63e12e1bef0fe72db44c4328681139ffca279e323e7ffbf9\": container with ID starting with 716dbcde722f4a3e63e12e1bef0fe72db44c4328681139ffca279e323e7ffbf9 not found: ID does not exist"
Sep 14 03:13:32.051611 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:32.051582    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:13:32.216471 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:32.216438    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89e994ea-c41a-4752-b6b3-00c18a4da521" path="/var/lib/kubelet/pods/89e994ea-c41a-4752-b6b3-00c18a4da521/volumes"
Sep 14 03:13:43.618250 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:43.618193    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh" podUID="604aed98-fc2c-4037-a9d5-3b383336e0f8" containerName="main" containerID="cri-o://89eb282be071c8fc6b193656e78e02dc8dad0df638b51ed985bdef7dac1ec478" gracePeriod=45
Sep 14 03:13:43.866527 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:43.866488    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:13:43.927653 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:43.927581    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-dshm\") pod \"604aed98-fc2c-4037-a9d5-3b383336e0f8\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") "
Sep 14 03:13:43.927653 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:43.927619    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/604aed98-fc2c-4037-a9d5-3b383336e0f8-tls-certs\") pod \"604aed98-fc2c-4037-a9d5-3b383336e0f8\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") "
Sep 14 03:13:43.927849 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:43.927664    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-model-cache\") pod \"604aed98-fc2c-4037-a9d5-3b383336e0f8\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") "
Sep 14 03:13:43.927849 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:43.927688    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-tmp-dir\") pod \"604aed98-fc2c-4037-a9d5-3b383336e0f8\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") "
Sep 14 03:13:43.927849 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:43.927723    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-home\") pod \"604aed98-fc2c-4037-a9d5-3b383336e0f8\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") "
Sep 14 03:13:43.927849 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:43.927744    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wkz9\" (UniqueName: \"kubernetes.io/projected/604aed98-fc2c-4037-a9d5-3b383336e0f8-kube-api-access-5wkz9\") pod \"604aed98-fc2c-4037-a9d5-3b383336e0f8\" (UID: \"604aed98-fc2c-4037-a9d5-3b383336e0f8\") "
Sep 14 03:13:43.928077 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:43.927954    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-model-cache" (OuterVolumeSpecName: "model-cache") pod "604aed98-fc2c-4037-a9d5-3b383336e0f8" (UID: "604aed98-fc2c-4037-a9d5-3b383336e0f8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:13:43.928077 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:43.928049    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-home" (OuterVolumeSpecName: "home") pod "604aed98-fc2c-4037-a9d5-3b383336e0f8" (UID: "604aed98-fc2c-4037-a9d5-3b383336e0f8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:13:43.928162 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:43.928064    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "604aed98-fc2c-4037-a9d5-3b383336e0f8" (UID: "604aed98-fc2c-4037-a9d5-3b383336e0f8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:13:43.929841 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:43.929822    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-dshm" (OuterVolumeSpecName: "dshm") pod "604aed98-fc2c-4037-a9d5-3b383336e0f8" (UID: "604aed98-fc2c-4037-a9d5-3b383336e0f8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:13:43.929990 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:43.929969    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/604aed98-fc2c-4037-a9d5-3b383336e0f8-kube-api-access-5wkz9" (OuterVolumeSpecName: "kube-api-access-5wkz9") pod "604aed98-fc2c-4037-a9d5-3b383336e0f8" (UID: "604aed98-fc2c-4037-a9d5-3b383336e0f8"). InnerVolumeSpecName "kube-api-access-5wkz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:13:43.930036 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:43.930003    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/604aed98-fc2c-4037-a9d5-3b383336e0f8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "604aed98-fc2c-4037-a9d5-3b383336e0f8" (UID: "604aed98-fc2c-4037-a9d5-3b383336e0f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:13:44.029255 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.029232    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:44.029255 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.029254    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/604aed98-fc2c-4037-a9d5-3b383336e0f8-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:44.029380 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.029263    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:44.029380 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.029271    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:44.029380 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.029279    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/604aed98-fc2c-4037-a9d5-3b383336e0f8-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:44.029380 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.029286    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wkz9\" (UniqueName: \"kubernetes.io/projected/604aed98-fc2c-4037-a9d5-3b383336e0f8-kube-api-access-5wkz9\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:44.079832 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.079804    2574 generic.go:356] "Generic (PLEG): container finished" podID="604aed98-fc2c-4037-a9d5-3b383336e0f8" containerID="89eb282be071c8fc6b193656e78e02dc8dad0df638b51ed985bdef7dac1ec478" exitCode=0
Sep 14 03:13:44.079925 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.079844    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"
Sep 14 03:13:44.079925 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.079883    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh" event={"ID":"604aed98-fc2c-4037-a9d5-3b383336e0f8","Type":"ContainerDied","Data":"89eb282be071c8fc6b193656e78e02dc8dad0df638b51ed985bdef7dac1ec478"}
Sep 14 03:13:44.079925 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.079920    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh" event={"ID":"604aed98-fc2c-4037-a9d5-3b383336e0f8","Type":"ContainerDied","Data":"8fc17c6b1a724a90adef2f03d94bfb27b9700d64afcac2252340e2bd6cffc2c4"}
Sep 14 03:13:44.080036 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.079936    2574 scope.go:117] "RemoveContainer" containerID="89eb282be071c8fc6b193656e78e02dc8dad0df638b51ed985bdef7dac1ec478"
Sep 14 03:13:44.088567 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.088550    2574 scope.go:117] "RemoveContainer" containerID="89eb282be071c8fc6b193656e78e02dc8dad0df638b51ed985bdef7dac1ec478"
Sep 14 03:13:44.088836 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:13:44.088818    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89eb282be071c8fc6b193656e78e02dc8dad0df638b51ed985bdef7dac1ec478\": container with ID starting with 89eb282be071c8fc6b193656e78e02dc8dad0df638b51ed985bdef7dac1ec478 not found: ID does not exist" containerID="89eb282be071c8fc6b193656e78e02dc8dad0df638b51ed985bdef7dac1ec478"
Sep 14 03:13:44.088901 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.088847    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89eb282be071c8fc6b193656e78e02dc8dad0df638b51ed985bdef7dac1ec478"} err="failed to get container status \"89eb282be071c8fc6b193656e78e02dc8dad0df638b51ed985bdef7dac1ec478\": rpc error: code = NotFound desc = could not find container \"89eb282be071c8fc6b193656e78e02dc8dad0df638b51ed985bdef7dac1ec478\": container with ID starting with 89eb282be071c8fc6b193656e78e02dc8dad0df638b51ed985bdef7dac1ec478 not found: ID does not exist"
Sep 14 03:13:44.101985 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.101956    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"]
Sep 14 03:13:44.107420 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.107400    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/sub-test-model-0c919e-kserve-55f8f4669-jrwdh"]
Sep 14 03:13:44.215905 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.215856    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="604aed98-fc2c-4037-a9d5-3b383336e0f8" path="/var/lib/kubelet/pods/604aed98-fc2c-4037-a9d5-3b383336e0f8/volumes"
Sep 14 03:13:44.516078 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.516045    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr" podUID="6dbe09a5-e402-43e4-b84b-788354eb3d37" containerName="main" containerID="cri-o://5fda7d3acee8e9079d2cac9c6726e70506d1f086c36db302ee4e4309ce408e7d" gracePeriod=45
Sep 14 03:13:44.768083 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.768031    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:13:44.835254 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.835229    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnjvq\" (UniqueName: \"kubernetes.io/projected/6dbe09a5-e402-43e4-b84b-788354eb3d37-kube-api-access-rnjvq\") pod \"6dbe09a5-e402-43e4-b84b-788354eb3d37\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") "
Sep 14 03:13:44.835391 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.835263    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-dshm\") pod \"6dbe09a5-e402-43e4-b84b-788354eb3d37\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") "
Sep 14 03:13:44.835391 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.835296    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-tmp-dir\") pod \"6dbe09a5-e402-43e4-b84b-788354eb3d37\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") "
Sep 14 03:13:44.835544 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.835414    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-home\") pod \"6dbe09a5-e402-43e4-b84b-788354eb3d37\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") "
Sep 14 03:13:44.835544 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.835460    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dbe09a5-e402-43e4-b84b-788354eb3d37-tls-certs\") pod \"6dbe09a5-e402-43e4-b84b-788354eb3d37\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") "
Sep 14 03:13:44.835544 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.835488    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-model-cache\") pod \"6dbe09a5-e402-43e4-b84b-788354eb3d37\" (UID: \"6dbe09a5-e402-43e4-b84b-788354eb3d37\") "
Sep 14 03:13:44.835701 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.835564    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6dbe09a5-e402-43e4-b84b-788354eb3d37" (UID: "6dbe09a5-e402-43e4-b84b-788354eb3d37"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:13:44.835701 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.835646    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-home" (OuterVolumeSpecName: "home") pod "6dbe09a5-e402-43e4-b84b-788354eb3d37" (UID: "6dbe09a5-e402-43e4-b84b-788354eb3d37"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:13:44.835795 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.835770    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-model-cache" (OuterVolumeSpecName: "model-cache") pod "6dbe09a5-e402-43e4-b84b-788354eb3d37" (UID: "6dbe09a5-e402-43e4-b84b-788354eb3d37"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:13:44.835918 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.835902    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:44.835967 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.835924    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:44.835967 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.835937    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:44.837611 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.837592    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dbe09a5-e402-43e4-b84b-788354eb3d37-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6dbe09a5-e402-43e4-b84b-788354eb3d37" (UID: "6dbe09a5-e402-43e4-b84b-788354eb3d37"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:13:44.838036 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.838007    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-dshm" (OuterVolumeSpecName: "dshm") pod "6dbe09a5-e402-43e4-b84b-788354eb3d37" (UID: "6dbe09a5-e402-43e4-b84b-788354eb3d37"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:13:44.838120 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.838009    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dbe09a5-e402-43e4-b84b-788354eb3d37-kube-api-access-rnjvq" (OuterVolumeSpecName: "kube-api-access-rnjvq") pod "6dbe09a5-e402-43e4-b84b-788354eb3d37" (UID: "6dbe09a5-e402-43e4-b84b-788354eb3d37"). InnerVolumeSpecName "kube-api-access-rnjvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:13:44.937062 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.937040    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dbe09a5-e402-43e4-b84b-788354eb3d37-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:44.937062 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.937061    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dbe09a5-e402-43e4-b84b-788354eb3d37-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:44.937214 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:44.937071    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnjvq\" (UniqueName: \"kubernetes.io/projected/6dbe09a5-e402-43e4-b84b-788354eb3d37-kube-api-access-rnjvq\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:45.086499 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.086422    2574 generic.go:356] "Generic (PLEG): container finished" podID="6dbe09a5-e402-43e4-b84b-788354eb3d37" containerID="5fda7d3acee8e9079d2cac9c6726e70506d1f086c36db302ee4e4309ce408e7d" exitCode=0
Sep 14 03:13:45.086499 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.086462    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"
Sep 14 03:13:45.086709 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.086507    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr" event={"ID":"6dbe09a5-e402-43e4-b84b-788354eb3d37","Type":"ContainerDied","Data":"5fda7d3acee8e9079d2cac9c6726e70506d1f086c36db302ee4e4309ce408e7d"}
Sep 14 03:13:45.086709 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.086560    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr" event={"ID":"6dbe09a5-e402-43e4-b84b-788354eb3d37","Type":"ContainerDied","Data":"fe3606eaea6156ab2bf7044457a89cc6a5310d6bde4350c34d2e8902ba5c4b28"}
Sep 14 03:13:45.086709 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.086579    2574 scope.go:117] "RemoveContainer" containerID="5fda7d3acee8e9079d2cac9c6726e70506d1f086c36db302ee4e4309ce408e7d"
Sep 14 03:13:45.098832 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.098815    2574 scope.go:117] "RemoveContainer" containerID="5fda7d3acee8e9079d2cac9c6726e70506d1f086c36db302ee4e4309ce408e7d"
Sep 14 03:13:45.099085 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:13:45.099069    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fda7d3acee8e9079d2cac9c6726e70506d1f086c36db302ee4e4309ce408e7d\": container with ID starting with 5fda7d3acee8e9079d2cac9c6726e70506d1f086c36db302ee4e4309ce408e7d not found: ID does not exist" containerID="5fda7d3acee8e9079d2cac9c6726e70506d1f086c36db302ee4e4309ce408e7d"
Sep 14 03:13:45.099128 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.099093    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fda7d3acee8e9079d2cac9c6726e70506d1f086c36db302ee4e4309ce408e7d"} err="failed to get container status \"5fda7d3acee8e9079d2cac9c6726e70506d1f086c36db302ee4e4309ce408e7d\": rpc error: code = NotFound desc = could not find container \"5fda7d3acee8e9079d2cac9c6726e70506d1f086c36db302ee4e4309ce408e7d\": container with ID starting with 5fda7d3acee8e9079d2cac9c6726e70506d1f086c36db302ee4e4309ce408e7d not found: ID does not exist"
Sep 14 03:13:45.111933 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.111910    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"]
Sep 14 03:13:45.113712 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.113694    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/sub-test-model-f83c87-kserve-559b5847f7-6rhnr"]
Sep 14 03:13:45.366889 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.366795    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d" podUID="9bef1150-42c0-465e-a2a2-480c69e7e9e5" containerName="main" containerID="cri-o://597461b9cc3b3aaaab202521d4d05ab04ebf182840630264d7e542f43610cf27" gracePeriod=45
Sep 14 03:13:45.606986 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.606964    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:13:45.638870 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.638827    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:13:45.642858 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.642836    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-dshm\") pod \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") "
Sep 14 03:13:45.642956 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.642866    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bef1150-42c0-465e-a2a2-480c69e7e9e5-tls-certs\") pod \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") "
Sep 14 03:13:45.642956 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.642955    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-tmp-dir\") pod \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") "
Sep 14 03:13:45.643088 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.642971    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhcsj\" (UniqueName: \"kubernetes.io/projected/9bef1150-42c0-465e-a2a2-480c69e7e9e5-kube-api-access-fhcsj\") pod \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") "
Sep 14 03:13:45.643088 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.643044    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-model-cache\") pod \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") "
Sep 14 03:13:45.643202 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.643109    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-home\") pod \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\" (UID: \"9bef1150-42c0-465e-a2a2-480c69e7e9e5\") "
Sep 14 03:13:45.643299 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.643270    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9bef1150-42c0-465e-a2a2-480c69e7e9e5" (UID: "9bef1150-42c0-465e-a2a2-480c69e7e9e5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:13:45.643469 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.643447    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-model-cache" (OuterVolumeSpecName: "model-cache") pod "9bef1150-42c0-465e-a2a2-480c69e7e9e5" (UID: "9bef1150-42c0-465e-a2a2-480c69e7e9e5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:13:45.643469 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.643460    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:45.643820 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.643651    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-home" (OuterVolumeSpecName: "home") pod "9bef1150-42c0-465e-a2a2-480c69e7e9e5" (UID: "9bef1150-42c0-465e-a2a2-480c69e7e9e5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:13:45.645347 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.645321    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bef1150-42c0-465e-a2a2-480c69e7e9e5-kube-api-access-fhcsj" (OuterVolumeSpecName: "kube-api-access-fhcsj") pod "9bef1150-42c0-465e-a2a2-480c69e7e9e5" (UID: "9bef1150-42c0-465e-a2a2-480c69e7e9e5"). InnerVolumeSpecName "kube-api-access-fhcsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:13:45.645461 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.645404    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bef1150-42c0-465e-a2a2-480c69e7e9e5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9bef1150-42c0-465e-a2a2-480c69e7e9e5" (UID: "9bef1150-42c0-465e-a2a2-480c69e7e9e5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:13:45.645461 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.645412    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-dshm" (OuterVolumeSpecName: "dshm") pod "9bef1150-42c0-465e-a2a2-480c69e7e9e5" (UID: "9bef1150-42c0-465e-a2a2-480c69e7e9e5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:13:45.744649 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.744624    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:45.744649 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.744647    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:45.744649 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.744656    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bef1150-42c0-465e-a2a2-480c69e7e9e5-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:45.744856 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.744663    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bef1150-42c0-465e-a2a2-480c69e7e9e5-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:45.744856 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:45.744671    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhcsj\" (UniqueName: \"kubernetes.io/projected/9bef1150-42c0-465e-a2a2-480c69e7e9e5-kube-api-access-fhcsj\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:13:46.095904 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:46.095877    2574 generic.go:356] "Generic (PLEG): container finished" podID="9bef1150-42c0-465e-a2a2-480c69e7e9e5" containerID="597461b9cc3b3aaaab202521d4d05ab04ebf182840630264d7e542f43610cf27" exitCode=0
Sep 14 03:13:46.096290 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:46.095918    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"
Sep 14 03:13:46.096290 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:46.095932    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d" event={"ID":"9bef1150-42c0-465e-a2a2-480c69e7e9e5","Type":"ContainerDied","Data":"597461b9cc3b3aaaab202521d4d05ab04ebf182840630264d7e542f43610cf27"}
Sep 14 03:13:46.096290 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:46.095961    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d" event={"ID":"9bef1150-42c0-465e-a2a2-480c69e7e9e5","Type":"ContainerDied","Data":"9696b8629c8deb0395cb191cd6173e4a32963529fd6b5e38107b4f8e30a101cb"}
Sep 14 03:13:46.096290 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:46.095977    2574 scope.go:117] "RemoveContainer" containerID="597461b9cc3b3aaaab202521d4d05ab04ebf182840630264d7e542f43610cf27"
Sep 14 03:13:46.105366 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:46.105349    2574 scope.go:117] "RemoveContainer" containerID="597461b9cc3b3aaaab202521d4d05ab04ebf182840630264d7e542f43610cf27"
Sep 14 03:13:46.105674 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:13:46.105656    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"597461b9cc3b3aaaab202521d4d05ab04ebf182840630264d7e542f43610cf27\": container with ID starting with 597461b9cc3b3aaaab202521d4d05ab04ebf182840630264d7e542f43610cf27 not found: ID does not exist" containerID="597461b9cc3b3aaaab202521d4d05ab04ebf182840630264d7e542f43610cf27"
Sep 14 03:13:46.105737 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:46.105682    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"597461b9cc3b3aaaab202521d4d05ab04ebf182840630264d7e542f43610cf27"} err="failed to get container status \"597461b9cc3b3aaaab202521d4d05ab04ebf182840630264d7e542f43610cf27\": rpc error: code = NotFound desc = could not find container \"597461b9cc3b3aaaab202521d4d05ab04ebf182840630264d7e542f43610cf27\": container with ID starting with 597461b9cc3b3aaaab202521d4d05ab04ebf182840630264d7e542f43610cf27 not found: ID does not exist"
Sep 14 03:13:46.116911 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:46.116887    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"]
Sep 14 03:13:46.120747 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:46.120727    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-d6ac6dc8/e2e-lifecycle-model-d6ac6dc8-kserve-6b66857bff-m5q8d"]
Sep 14 03:13:46.216005 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:46.215981    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dbe09a5-e402-43e4-b84b-788354eb3d37" path="/var/lib/kubelet/pods/6dbe09a5-e402-43e4-b84b-788354eb3d37/volumes"
Sep 14 03:13:46.216357 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:46.216344    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bef1150-42c0-465e-a2a2-480c69e7e9e5" path="/var/lib/kubelet/pods/9bef1150-42c0-465e-a2a2-480c69e7e9e5/volumes"
Sep 14 03:13:50.398201 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.398163    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"]
Sep 14 03:13:50.398602 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.398545    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="604aed98-fc2c-4037-a9d5-3b383336e0f8" containerName="main"
Sep 14 03:13:50.398602 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.398557    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="604aed98-fc2c-4037-a9d5-3b383336e0f8" containerName="main"
Sep 14 03:13:50.398602 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.398571    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bef1150-42c0-465e-a2a2-480c69e7e9e5" containerName="main"
Sep 14 03:13:50.398602 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.398576    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bef1150-42c0-465e-a2a2-480c69e7e9e5" containerName="main"
Sep 14 03:13:50.398602 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.398587    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89e994ea-c41a-4752-b6b3-00c18a4da521" containerName="test-curl-8996-c5633a"
Sep 14 03:13:50.398602 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.398592    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="89e994ea-c41a-4752-b6b3-00c18a4da521" containerName="test-curl-8996-c5633a"
Sep 14 03:13:50.398839 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.398607    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dbe09a5-e402-43e4-b84b-788354eb3d37" containerName="main"
Sep 14 03:13:50.398839 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.398612    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dbe09a5-e402-43e4-b84b-788354eb3d37" containerName="main"
Sep 14 03:13:50.398839 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.398679    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="89e994ea-c41a-4752-b6b3-00c18a4da521" containerName="test-curl-8996-c5633a"
Sep 14 03:13:50.398839 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.398686    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bef1150-42c0-465e-a2a2-480c69e7e9e5" containerName="main"
Sep 14 03:13:50.398839 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.398692    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dbe09a5-e402-43e4-b84b-788354eb3d37" containerName="main"
Sep 14 03:13:50.398839 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.398698    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="604aed98-fc2c-4037-a9d5-3b383336e0f8" containerName="main"
Sep 14 03:13:50.403199 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.403177    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.407634 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.407609    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-0c919e\"/\"openshift-service-ca.crt\""
Sep 14 03:13:50.407749 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.407611    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-0c919e\"/\"rate-test-model-0c919e-kserve-self-signed-certs\""
Sep 14 03:13:50.407925 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.407909    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-0c919e\"/\"default-dockercfg-llp22\""
Sep 14 03:13:50.407970 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.407909    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-0c919e\"/\"kube-root-ca.crt\""
Sep 14 03:13:50.414026 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.414005    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"]
Sep 14 03:13:50.484281 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.484245    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-dshm\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.484458 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.484342    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-model-cache\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.484458 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.484359    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9png\" (UniqueName: \"kubernetes.io/projected/1277df0a-0d9d-4cd5-9530-70f23c8438a5-kube-api-access-p9png\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.484458 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.484392    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-home\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.484458 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.484408    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-tmp-dir\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.484458 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.484422    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1277df0a-0d9d-4cd5-9530-70f23c8438a5-tls-certs\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.585156 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.585126    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-dshm\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.585312 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.585195    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-model-cache\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.585312 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.585213    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9png\" (UniqueName: \"kubernetes.io/projected/1277df0a-0d9d-4cd5-9530-70f23c8438a5-kube-api-access-p9png\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.585312 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.585244    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-home\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.585312 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.585260    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-tmp-dir\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.585312 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.585276    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1277df0a-0d9d-4cd5-9530-70f23c8438a5-tls-certs\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.585689 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.585660    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-home\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.585689 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.585686    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-tmp-dir\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.585875 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.585736    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-model-cache\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.587685 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.587664    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-dshm\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.587851 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.587834    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1277df0a-0d9d-4cd5-9530-70f23c8438a5-tls-certs\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.595764 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.595738    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9png\" (UniqueName: \"kubernetes.io/projected/1277df0a-0d9d-4cd5-9530-70f23c8438a5-kube-api-access-p9png\") pod \"rate-test-model-0c919e-kserve-77f4794c79-nn5c8\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") " pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.712445 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.712354    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:13:50.988881 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:50.988858    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"]
Sep 14 03:13:50.990771 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:13:50.990738    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1277df0a_0d9d_4cd5_9530_70f23c8438a5.slice/crio-4427d9d1d62faf751867807a5991a322867e926f351f69a109cab0c83469c646 WatchSource:0}: Error finding container 4427d9d1d62faf751867807a5991a322867e926f351f69a109cab0c83469c646: Status 404 returned error can't find the container with id 4427d9d1d62faf751867807a5991a322867e926f351f69a109cab0c83469c646
Sep 14 03:13:51.124577 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:51.124543    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8" event={"ID":"1277df0a-0d9d-4cd5-9530-70f23c8438a5","Type":"ContainerStarted","Data":"775057f37ec1dcd1652af1b1732784805a1fc71b4cc3ac42649d71b76218460c"}
Sep 14 03:13:51.124730 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:51.124584    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8" event={"ID":"1277df0a-0d9d-4cd5-9530-70f23c8438a5","Type":"ContainerStarted","Data":"4427d9d1d62faf751867807a5991a322867e926f351f69a109cab0c83469c646"}
Sep 14 03:13:51.144545 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:51.144486    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8" podStartSLOduration=1.1444702549999999 podStartE2EDuration="1.144470255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:13:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:13:51.142636711 +0000 UTC m=+1377.455660532" watchObservedRunningTime="2026-09-14 03:13:51.144470255 +0000 UTC m=+1377.457494075"
Sep 14 03:13:54.340592 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:54.340562    2574 scope.go:117] "RemoveContainer" containerID="ada941ae22fabc81bfb1af81566bc9d18be2ad6f1a6b5f935e110304d565e62b"
Sep 14 03:13:54.348961 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:54.348938    2574 scope.go:117] "RemoveContainer" containerID="a6f2713ebb6786284413bc68ed307a67954374af912b9f22b8ebe0efb9eb735c"
Sep 14 03:13:54.356701 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:54.356684    2574 scope.go:117] "RemoveContainer" containerID="e32f69f8d47ba5a0b04f49004665753f1a983c827dde9383725e1ae185d73fa3"
Sep 14 03:13:54.739225 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:13:54.739197    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:14:00.712616 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:00.712581    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:14:00.713133 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:00.712862    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:14:00.725489 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:00.725461    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:14:01.191254 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:01.191232    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:14:14.482231 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:14.482195    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"]
Sep 14 03:14:14.482717 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:14.482454    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz" podUID="b3bf449c-92d9-42a0-a589-85fb9862e51a" containerName="istio-proxy" containerID="cri-o://f8035f755b1818c652ccf26ddccfca3363cb87dc09383a6485bfe24235897763" gracePeriod=30
Sep 14 03:14:15.288682 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.288642    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"]
Sep 14 03:14:15.293933 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.293912    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.298249 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.298229    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-f83c87\"/\"rate-test-model-f83c87-kserve-self-signed-certs\""
Sep 14 03:14:15.298249 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.298247    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-f83c87\"/\"kube-root-ca.crt\""
Sep 14 03:14:15.298415 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.298229    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-f83c87\"/\"openshift-service-ca.crt\""
Sep 14 03:14:15.298415 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.298280    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-f83c87\"/\"default-dockercfg-j84w5\""
Sep 14 03:14:15.303441 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.303422    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"]
Sep 14 03:14:15.397926 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.397901    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-dshm\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.398033 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.397938    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-tmp-dir\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.398033 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.397955    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-home\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.398033 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.398024    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkr5r\" (UniqueName: \"kubernetes.io/projected/7535bc6b-9712-412a-a91c-3013cd07b520-kube-api-access-nkr5r\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.398145 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.398076    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-model-cache\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.398145 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.398129    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7535bc6b-9712-412a-a91c-3013cd07b520-tls-certs\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.499019 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.498979    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-dshm\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.499429 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.499031    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-tmp-dir\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.499429 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.499050    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-home\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.499429 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.499083    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkr5r\" (UniqueName: \"kubernetes.io/projected/7535bc6b-9712-412a-a91c-3013cd07b520-kube-api-access-nkr5r\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.499429 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.499100    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-model-cache\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.499429 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.499143    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7535bc6b-9712-412a-a91c-3013cd07b520-tls-certs\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.499694 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.499478    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-home\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.499694 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.499545    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-model-cache\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.499694 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.499605    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-tmp-dir\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.501504 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.501483    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-dshm\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.501792 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.501776    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7535bc6b-9712-412a-a91c-3013cd07b520-tls-certs\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.508061 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.508028    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkr5r\" (UniqueName: \"kubernetes.io/projected/7535bc6b-9712-412a-a91c-3013cd07b520-kube-api-access-nkr5r\") pod \"rate-test-model-f83c87-kserve-7579f9887-dm9br\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") " pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.603542 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.603459    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:15.875600 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:15.875574    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"]
Sep 14 03:14:15.877181 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:14:15.877154    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7535bc6b_9712_412a_a91c_3013cd07b520.slice/crio-f0fc00e5f0ad0a4399555a35488456728665c31752b337c5023bfd87fd7403f3 WatchSource:0}: Error finding container f0fc00e5f0ad0a4399555a35488456728665c31752b337c5023bfd87fd7403f3: Status 404 returned error can't find the container with id f0fc00e5f0ad0a4399555a35488456728665c31752b337c5023bfd87fd7403f3
Sep 14 03:14:16.253809 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:16.253775    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br" event={"ID":"7535bc6b-9712-412a-a91c-3013cd07b520","Type":"ContainerStarted","Data":"12bb374e4c911dcfb0287e0f9e9a890313096c618c33396df63fe56e1e1ecf78"}
Sep 14 03:14:16.253967 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:16.253817    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br" event={"ID":"7535bc6b-9712-412a-a91c-3013cd07b520","Type":"ContainerStarted","Data":"f0fc00e5f0ad0a4399555a35488456728665c31752b337c5023bfd87fd7403f3"}
Sep 14 03:14:16.274122 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:16.274076    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br" podStartSLOduration=1.274061571 podStartE2EDuration="1.274061571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:14:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:14:16.27277747 +0000 UTC m=+1402.585801289" watchObservedRunningTime="2026-09-14 03:14:16.274061571 +0000 UTC m=+1402.587085451"
Sep 14 03:14:19.733734 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.733711    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:14:19.839825 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.839771    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3bf449c-92d9-42a0-a589-85fb9862e51a-istiod-ca-cert\") pod \"b3bf449c-92d9-42a0-a589-85fb9862e51a\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") "
Sep 14 03:14:19.839825 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.839813    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpp44\" (UniqueName: \"kubernetes.io/projected/b3bf449c-92d9-42a0-a589-85fb9862e51a-kube-api-access-cpp44\") pod \"b3bf449c-92d9-42a0-a589-85fb9862e51a\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") "
Sep 14 03:14:19.839968 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.839838    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-credential-socket\") pod \"b3bf449c-92d9-42a0-a589-85fb9862e51a\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") "
Sep 14 03:14:19.839968 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.839889    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-podinfo\") pod \"b3bf449c-92d9-42a0-a589-85fb9862e51a\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") "
Sep 14 03:14:19.839968 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.839929    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-token\") pod \"b3bf449c-92d9-42a0-a589-85fb9862e51a\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") "
Sep 14 03:14:19.840108 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.839975    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-envoy\") pod \"b3bf449c-92d9-42a0-a589-85fb9862e51a\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") "
Sep 14 03:14:19.840108 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.840022    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-data\") pod \"b3bf449c-92d9-42a0-a589-85fb9862e51a\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") "
Sep 14 03:14:19.840108 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.840061    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-workload-socket\") pod \"b3bf449c-92d9-42a0-a589-85fb9862e51a\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") "
Sep 14 03:14:19.840108 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.840100    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-workload-certs\") pod \"b3bf449c-92d9-42a0-a589-85fb9862e51a\" (UID: \"b3bf449c-92d9-42a0-a589-85fb9862e51a\") "
Sep 14 03:14:19.840306 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.840171    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3bf449c-92d9-42a0-a589-85fb9862e51a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b3bf449c-92d9-42a0-a589-85fb9862e51a" (UID: "b3bf449c-92d9-42a0-a589-85fb9862e51a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:14:19.840306 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.840173    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b3bf449c-92d9-42a0-a589-85fb9862e51a" (UID: "b3bf449c-92d9-42a0-a589-85fb9862e51a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:14:19.840428 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.840306    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-data" (OuterVolumeSpecName: "istio-data") pod "b3bf449c-92d9-42a0-a589-85fb9862e51a" (UID: "b3bf449c-92d9-42a0-a589-85fb9862e51a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:14:19.840488 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.840440    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b3bf449c-92d9-42a0-a589-85fb9862e51a" (UID: "b3bf449c-92d9-42a0-a589-85fb9862e51a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:14:19.840488 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.840455    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b3bf449c-92d9-42a0-a589-85fb9862e51a" (UID: "b3bf449c-92d9-42a0-a589-85fb9862e51a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:14:19.840628 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.840529    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3bf449c-92d9-42a0-a589-85fb9862e51a-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:14:19.840628 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.840550    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:14:19.840628 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.840591    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:14:19.842394 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.842365    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b3bf449c-92d9-42a0-a589-85fb9862e51a" (UID: "b3bf449c-92d9-42a0-a589-85fb9862e51a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:14:19.842489 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.842358    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b3bf449c-92d9-42a0-a589-85fb9862e51a" (UID: "b3bf449c-92d9-42a0-a589-85fb9862e51a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:14:19.842489 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.842396    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3bf449c-92d9-42a0-a589-85fb9862e51a-kube-api-access-cpp44" (OuterVolumeSpecName: "kube-api-access-cpp44") pod "b3bf449c-92d9-42a0-a589-85fb9862e51a" (UID: "b3bf449c-92d9-42a0-a589-85fb9862e51a"). InnerVolumeSpecName "kube-api-access-cpp44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:14:19.842633 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.842614    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-token" (OuterVolumeSpecName: "istio-token") pod "b3bf449c-92d9-42a0-a589-85fb9862e51a" (UID: "b3bf449c-92d9-42a0-a589-85fb9862e51a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:14:19.941857 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.941832    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:14:19.941857 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.941856    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpp44\" (UniqueName: \"kubernetes.io/projected/b3bf449c-92d9-42a0-a589-85fb9862e51a-kube-api-access-cpp44\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:14:19.942033 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.941867    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:14:19.942033 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.941875    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:14:19.942033 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.941883    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:14:19.942033 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:19.941891    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3bf449c-92d9-42a0-a589-85fb9862e51a-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:14:20.039241 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:20.039215    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:14:20.275477 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:20.275453    2574 generic.go:356] "Generic (PLEG): container finished" podID="b3bf449c-92d9-42a0-a589-85fb9862e51a" containerID="f8035f755b1818c652ccf26ddccfca3363cb87dc09383a6485bfe24235897763" exitCode=0
Sep 14 03:14:20.275609 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:20.275507    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"
Sep 14 03:14:20.275609 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:20.275552    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz" event={"ID":"b3bf449c-92d9-42a0-a589-85fb9862e51a","Type":"ContainerDied","Data":"f8035f755b1818c652ccf26ddccfca3363cb87dc09383a6485bfe24235897763"}
Sep 14 03:14:20.275609 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:20.275598    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz" event={"ID":"b3bf449c-92d9-42a0-a589-85fb9862e51a","Type":"ContainerDied","Data":"564655d6fc900ef542ab3768a935e63a3586114be882a12c4feca41955875fea"}
Sep 14 03:14:20.275733 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:20.275619    2574 scope.go:117] "RemoveContainer" containerID="f8035f755b1818c652ccf26ddccfca3363cb87dc09383a6485bfe24235897763"
Sep 14 03:14:20.284339 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:20.284323    2574 scope.go:117] "RemoveContainer" containerID="bb0037605705cd19aa640c220f1001e0e2caea74aa9f4a59586b741bb3fe4e5b"
Sep 14 03:14:20.292744 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:20.292728    2574 scope.go:117] "RemoveContainer" containerID="f8035f755b1818c652ccf26ddccfca3363cb87dc09383a6485bfe24235897763"
Sep 14 03:14:20.293072 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:14:20.293038    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8035f755b1818c652ccf26ddccfca3363cb87dc09383a6485bfe24235897763\": container with ID starting with f8035f755b1818c652ccf26ddccfca3363cb87dc09383a6485bfe24235897763 not found: ID does not exist" containerID="f8035f755b1818c652ccf26ddccfca3363cb87dc09383a6485bfe24235897763"
Sep 14 03:14:20.293167 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:20.293084    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8035f755b1818c652ccf26ddccfca3363cb87dc09383a6485bfe24235897763"} err="failed to get container status \"f8035f755b1818c652ccf26ddccfca3363cb87dc09383a6485bfe24235897763\": rpc error: code = NotFound desc = could not find container \"f8035f755b1818c652ccf26ddccfca3363cb87dc09383a6485bfe24235897763\": container with ID starting with f8035f755b1818c652ccf26ddccfca3363cb87dc09383a6485bfe24235897763 not found: ID does not exist"
Sep 14 03:14:20.293167 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:20.293109    2574 scope.go:117] "RemoveContainer" containerID="bb0037605705cd19aa640c220f1001e0e2caea74aa9f4a59586b741bb3fe4e5b"
Sep 14 03:14:20.293487 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:14:20.293468    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb0037605705cd19aa640c220f1001e0e2caea74aa9f4a59586b741bb3fe4e5b\": container with ID starting with bb0037605705cd19aa640c220f1001e0e2caea74aa9f4a59586b741bb3fe4e5b not found: ID does not exist" containerID="bb0037605705cd19aa640c220f1001e0e2caea74aa9f4a59586b741bb3fe4e5b"
Sep 14 03:14:20.293634 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:20.293492    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb0037605705cd19aa640c220f1001e0e2caea74aa9f4a59586b741bb3fe4e5b"} err="failed to get container status \"bb0037605705cd19aa640c220f1001e0e2caea74aa9f4a59586b741bb3fe4e5b\": rpc error: code = NotFound desc = could not find container \"bb0037605705cd19aa640c220f1001e0e2caea74aa9f4a59586b741bb3fe4e5b\": container with ID starting with bb0037605705cd19aa640c220f1001e0e2caea74aa9f4a59586b741bb3fe4e5b not found: ID does not exist"
Sep 14 03:14:20.295013 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:20.294990    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"]
Sep 14 03:14:20.301035 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:20.301016    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz"]
Sep 14 03:14:22.215808 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:22.215768    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3bf449c-92d9-42a0-a589-85fb9862e51a" path="/var/lib/kubelet/pods/b3bf449c-92d9-42a0-a589-85fb9862e51a/volumes"
Sep 14 03:14:25.603902 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:25.603869    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:25.604316 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:25.603929    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:25.622819 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:25.622785    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:26.317182 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:26.317153    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:14:35.730405 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:35.730374    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:14:54.935203 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:14:54.935167    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:15:00.160502 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.160468    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822595-295td"]
Sep 14 03:15:00.160925 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.160838    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3bf449c-92d9-42a0-a589-85fb9862e51a" containerName="istio-proxy"
Sep 14 03:15:00.160925 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.160850    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3bf449c-92d9-42a0-a589-85fb9862e51a" containerName="istio-proxy"
Sep 14 03:15:00.160925 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.160864    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3bf449c-92d9-42a0-a589-85fb9862e51a" containerName="istio-proxy"
Sep 14 03:15:00.160925 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.160869    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3bf449c-92d9-42a0-a589-85fb9862e51a" containerName="istio-proxy"
Sep 14 03:15:00.160925 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.160926    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3bf449c-92d9-42a0-a589-85fb9862e51a" containerName="istio-proxy"
Sep 14 03:15:00.161151 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.160939    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3bf449c-92d9-42a0-a589-85fb9862e51a" containerName="istio-proxy"
Sep 14 03:15:00.165131 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.165114    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-295td"
Sep 14 03:15:00.168601 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.168581    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4hf5p\""
Sep 14 03:15:00.175743 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.175724    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-295td"
Sep 14 03:15:00.178069 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.178046    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822595-295td"]
Sep 14 03:15:00.181946 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.181925    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f83c87-29822595-xbb48"]
Sep 14 03:15:00.189148 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.188940    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-6e87d0-29822595-sdb6m"]
Sep 14 03:15:00.190119 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.190097    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f83c87-29822595-xbb48"
Sep 14 03:15:00.202851 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.202818    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-6e87d0-29822595-sdb6m"
Sep 14 03:15:00.203372 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.203299    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-6e87d0-29822595-sdb6m"]
Sep 14 03:15:00.226576 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.226548    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f83c87-29822595-xbb48"]
Sep 14 03:15:00.226892 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.226870    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f83c87-29822595-xbb48"
Sep 14 03:15:00.235644 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.235617    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-6e87d0-29822595-sdb6m"
Sep 14 03:15:00.466389 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.466363    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822595-295td"]
Sep 14 03:15:00.467840 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:15:00.467812    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a38cff9_eeac_4934_9e0a_989fcf3b0b2d.slice/crio-9142176951f00b1c025106cc35b6aa47d06fd25e13e0f975907ba79c8aab6ab4 WatchSource:0}: Error finding container 9142176951f00b1c025106cc35b6aa47d06fd25e13e0f975907ba79c8aab6ab4: Status 404 returned error can't find the container with id 9142176951f00b1c025106cc35b6aa47d06fd25e13e0f975907ba79c8aab6ab4
Sep 14 03:15:00.481564 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.481540    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-295td" event={"ID":"7a38cff9-eeac-4934-9e0a-989fcf3b0b2d","Type":"ContainerStarted","Data":"9142176951f00b1c025106cc35b6aa47d06fd25e13e0f975907ba79c8aab6ab4"}
Sep 14 03:15:00.520881 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.520857    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f83c87-29822595-xbb48"]
Sep 14 03:15:00.523692 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:15:00.523668    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8877cfcf_9755_421f_8607_d9abed3690a1.slice/crio-b77392042ef46524027d05ff212824881e28dd60f39f49e4e49f326495cf7e36 WatchSource:0}: Error finding container b77392042ef46524027d05ff212824881e28dd60f39f49e4e49f326495cf7e36: Status 404 returned error can't find the container with id b77392042ef46524027d05ff212824881e28dd60f39f49e4e49f326495cf7e36
Sep 14 03:15:00.537247 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:00.537226    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-6e87d0-29822595-sdb6m"]
Sep 14 03:15:00.537700 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:15:00.537679    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14ec788a_5fc5_4553_b94c_54a4ce6ae306.slice/crio-f5293fd7efe615f7b4455233047aef36f9df7ad3f89d1b1315c1ab5a264e4feb WatchSource:0}: Error finding container f5293fd7efe615f7b4455233047aef36f9df7ad3f89d1b1315c1ab5a264e4feb: Status 404 returned error can't find the container with id f5293fd7efe615f7b4455233047aef36f9df7ad3f89d1b1315c1ab5a264e4feb
Sep 14 03:15:01.088417 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:01.088332    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"]
Sep 14 03:15:01.488396 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:01.488363    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-6e87d0-29822595-sdb6m" event={"ID":"14ec788a-5fc5-4553-b94c-54a4ce6ae306","Type":"ContainerStarted","Data":"fd7d7ff002fed781a2a73149279eceae83f4875a274b4b0279c58b91335e554a"}
Sep 14 03:15:01.488755 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:01.488425    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-6e87d0-29822595-sdb6m" event={"ID":"14ec788a-5fc5-4553-b94c-54a4ce6ae306","Type":"ContainerStarted","Data":"f5293fd7efe615f7b4455233047aef36f9df7ad3f89d1b1315c1ab5a264e4feb"}
Sep 14 03:15:01.489947 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:01.489925    2574 generic.go:356] "Generic (PLEG): container finished" podID="7a38cff9-eeac-4934-9e0a-989fcf3b0b2d" containerID="1a68aafdd921f31669881e525f217393df40eaa1fcc0c35e6211ded56cd27572" exitCode=0
Sep 14 03:15:01.490031 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:01.489988    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-295td" event={"ID":"7a38cff9-eeac-4934-9e0a-989fcf3b0b2d","Type":"ContainerDied","Data":"1a68aafdd921f31669881e525f217393df40eaa1fcc0c35e6211ded56cd27572"}
Sep 14 03:15:01.491420 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:01.491399    2574 generic.go:356] "Generic (PLEG): container finished" podID="8877cfcf-9755-421f-8607-d9abed3690a1" containerID="63c80d54c9e53eca60f89ccf2c0d6051c52893f82210490862ab120a8d36ab66" exitCode=0
Sep 14 03:15:01.491502 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:01.491435    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f83c87-29822595-xbb48" event={"ID":"8877cfcf-9755-421f-8607-d9abed3690a1","Type":"ContainerDied","Data":"63c80d54c9e53eca60f89ccf2c0d6051c52893f82210490862ab120a8d36ab66"}
Sep 14 03:15:01.491502 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:01.491473    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f83c87-29822595-xbb48" event={"ID":"8877cfcf-9755-421f-8607-d9abed3690a1","Type":"ContainerStarted","Data":"b77392042ef46524027d05ff212824881e28dd60f39f49e4e49f326495cf7e36"}
Sep 14 03:15:01.504948 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:01.504909    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-6e87d0-29822595-sdb6m" podStartSLOduration=1.504897929 podStartE2EDuration="1.504897929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:01.503077507 +0000 UTC m=+1447.816101327" watchObservedRunningTime="2026-09-14 03:15:01.504897929 +0000 UTC m=+1447.817921751"
Sep 14 03:15:02.086397 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.086365    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"]
Sep 14 03:15:02.498032 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.498000    2574 generic.go:356] "Generic (PLEG): container finished" podID="14ec788a-5fc5-4553-b94c-54a4ce6ae306" containerID="fd7d7ff002fed781a2a73149279eceae83f4875a274b4b0279c58b91335e554a" exitCode=0
Sep 14 03:15:02.498538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.498097    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-6e87d0-29822595-sdb6m" event={"ID":"14ec788a-5fc5-4553-b94c-54a4ce6ae306","Type":"ContainerDied","Data":"fd7d7ff002fed781a2a73149279eceae83f4875a274b4b0279c58b91335e554a"}
Sep 14 03:15:02.632241 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.632210    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f83c87-29822595-xbb48"
Sep 14 03:15:02.666464 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.666436    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-295td"
Sep 14 03:15:02.871406 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.871328    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"]
Sep 14 03:15:02.871680 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.871667    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8877cfcf-9755-421f-8607-d9abed3690a1" containerName="cleanup"
Sep 14 03:15:02.871737 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.871682    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8877cfcf-9755-421f-8607-d9abed3690a1" containerName="cleanup"
Sep 14 03:15:02.871737 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.871696    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a38cff9-eeac-4934-9e0a-989fcf3b0b2d" containerName="cleanup"
Sep 14 03:15:02.871737 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.871702    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a38cff9-eeac-4934-9e0a-989fcf3b0b2d" containerName="cleanup"
Sep 14 03:15:02.871836 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.871773    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a38cff9-eeac-4934-9e0a-989fcf3b0b2d" containerName="cleanup"
Sep 14 03:15:02.871836 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.871782    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8877cfcf-9755-421f-8607-d9abed3690a1" containerName="cleanup"
Sep 14 03:15:02.874651 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.874636    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:02.878671 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.878654    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-2611af-openshift-default-dockercfg-xdrrr\""
Sep 14 03:15:02.892056 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.892009    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"]
Sep 14 03:15:02.997273 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.997242    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-workload-socket\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:02.997456 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.997295    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-data\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:02.997456 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.997320    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-podinfo\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:02.997456 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.997365    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5159351-6b1d-4ddd-a385-e7ea69d07591-istiod-ca-cert\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:02.997456 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.997394    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-token\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:02.997728 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.997471    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8lrv\" (UniqueName: \"kubernetes.io/projected/a5159351-6b1d-4ddd-a385-e7ea69d07591-kube-api-access-d8lrv\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:02.997728 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.997533    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-workload-certs\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:02.997728 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.997587    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-credential-socket\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:02.997728 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:02.997640    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-envoy\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.098894 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.098859    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-workload-socket\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.099041 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.098907    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-data\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.099041 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.098932    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-podinfo\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.099041 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.098958    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5159351-6b1d-4ddd-a385-e7ea69d07591-istiod-ca-cert\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.099041 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.098990    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-token\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.099041 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.099025    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8lrv\" (UniqueName: \"kubernetes.io/projected/a5159351-6b1d-4ddd-a385-e7ea69d07591-kube-api-access-d8lrv\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.099332 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.099158    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-workload-certs\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.099332 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.099257    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-credential-socket\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.099332 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.099303    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-envoy\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.099332 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.099316    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-data\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.099563 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.099369    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-workload-socket\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.099664 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.099642    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-workload-certs\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.099726 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.099703    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-credential-socket\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.099800 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.099782    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5159351-6b1d-4ddd-a385-e7ea69d07591-istiod-ca-cert\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.101493 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.101474    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-envoy\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.101580 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.101565    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-podinfo\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.108133 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.108109    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-token\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.108814 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.108793    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8lrv\" (UniqueName: \"kubernetes.io/projected/a5159351-6b1d-4ddd-a385-e7ea69d07591-kube-api-access-d8lrv\") pod \"e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") " pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.183966 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.183916    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:03.465810 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.465738    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"]
Sep 14 03:15:03.468917 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:15:03.468887    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5159351_6b1d_4ddd_a385_e7ea69d07591.slice/crio-be20ccd065ed33efaf572b8d0e67fd349d4d08a7fab0f39ca218e2d7b3beb9a1 WatchSource:0}: Error finding container be20ccd065ed33efaf572b8d0e67fd349d4d08a7fab0f39ca218e2d7b3beb9a1: Status 404 returned error can't find the container with id be20ccd065ed33efaf572b8d0e67fd349d4d08a7fab0f39ca218e2d7b3beb9a1
Sep 14 03:15:03.471231 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.471202    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:03.471295 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.471263    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:03.471295 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.471288    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:03.504197 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.504164    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-295td" event={"ID":"7a38cff9-eeac-4934-9e0a-989fcf3b0b2d","Type":"ContainerDied","Data":"9142176951f00b1c025106cc35b6aa47d06fd25e13e0f975907ba79c8aab6ab4"}
Sep 14 03:15:03.504542 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.504228    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9142176951f00b1c025106cc35b6aa47d06fd25e13e0f975907ba79c8aab6ab4"
Sep 14 03:15:03.504542 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.504178    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-295td"
Sep 14 03:15:03.505672 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.505653    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f83c87-29822595-xbb48"
Sep 14 03:15:03.505786 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.505688    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f83c87-29822595-xbb48" event={"ID":"8877cfcf-9755-421f-8607-d9abed3690a1","Type":"ContainerDied","Data":"b77392042ef46524027d05ff212824881e28dd60f39f49e4e49f326495cf7e36"}
Sep 14 03:15:03.505786 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.505733    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b77392042ef46524027d05ff212824881e28dd60f39f49e4e49f326495cf7e36"
Sep 14 03:15:03.507094 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.507071    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z" event={"ID":"a5159351-6b1d-4ddd-a385-e7ea69d07591","Type":"ContainerStarted","Data":"be20ccd065ed33efaf572b8d0e67fd349d4d08a7fab0f39ca218e2d7b3beb9a1"}
Sep 14 03:15:03.649654 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:03.649628    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-6e87d0-29822595-sdb6m"
Sep 14 03:15:04.513982 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:04.513954    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-6e87d0-29822595-sdb6m"
Sep 14 03:15:04.514390 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:04.513996    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-6e87d0-29822595-sdb6m" event={"ID":"14ec788a-5fc5-4553-b94c-54a4ce6ae306","Type":"ContainerDied","Data":"f5293fd7efe615f7b4455233047aef36f9df7ad3f89d1b1315c1ab5a264e4feb"}
Sep 14 03:15:04.514390 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:04.514046    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5293fd7efe615f7b4455233047aef36f9df7ad3f89d1b1315c1ab5a264e4feb"
Sep 14 03:15:04.515884 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:04.515858    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z" event={"ID":"a5159351-6b1d-4ddd-a385-e7ea69d07591","Type":"ContainerStarted","Data":"92b132094e3289191e00761fe19caf53a48426635013d37f306b610946c025ae"}
Sep 14 03:15:04.541085 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:04.541044    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z" podStartSLOduration=2.541030523 podStartE2EDuration="2.541030523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:04.537882753 +0000 UTC m=+1450.850906583" watchObservedRunningTime="2026-09-14 03:15:04.541030523 +0000 UTC m=+1450.854054394"
Sep 14 03:15:05.184896 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:05.184864    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:05.190003 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:05.189975    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:05.520993 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:05.520957    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:05.521897 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:05.521881    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:05.868375 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:05.868295    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:15:10.171438 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.171402    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"]
Sep 14 03:15:10.172032 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.171803    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14ec788a-5fc5-4553-b94c-54a4ce6ae306" containerName="cleanup"
Sep 14 03:15:10.172032 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.171817    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="14ec788a-5fc5-4553-b94c-54a4ce6ae306" containerName="cleanup"
Sep 14 03:15:10.172032 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.171897    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="14ec788a-5fc5-4553-b94c-54a4ce6ae306" containerName="cleanup"
Sep 14 03:15:10.175259 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.175236    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.178098 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.178074    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-40c586-openshift-default-dockercfg-cnbn2\""
Sep 14 03:15:10.188065 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.188041    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"]
Sep 14 03:15:10.262975 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.262946    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-istio-envoy\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.263125 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.263002    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-workload-certs\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.263125 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.263067    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f26433d-4127-42a6-b23c-86d89534305b-istio-token\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.263125 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.263112    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-istio-data\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.263252 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.263129    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmppn\" (UniqueName: \"kubernetes.io/projected/4f26433d-4127-42a6-b23c-86d89534305b-kube-api-access-zmppn\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.263252 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.263162    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-workload-socket\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.263252 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.263188    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-credential-socket\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.263252 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.263203    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f26433d-4127-42a6-b23c-86d89534305b-istiod-ca-cert\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.263252 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.263219    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f26433d-4127-42a6-b23c-86d89534305b-istio-podinfo\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.364561 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.364534    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-istio-envoy\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.364728 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.364586    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-workload-certs\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.364864 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.364784    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f26433d-4127-42a6-b23c-86d89534305b-istio-token\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.364941 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.364906    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-istio-data\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.365000 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.364939    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmppn\" (UniqueName: \"kubernetes.io/projected/4f26433d-4127-42a6-b23c-86d89534305b-kube-api-access-zmppn\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.365127 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.365100    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-workload-socket\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.365226 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.365166    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-credential-socket\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.365284 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.365228    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-istio-data\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.365284 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.365233    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f26433d-4127-42a6-b23c-86d89534305b-istiod-ca-cert\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.365284 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.365268    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f26433d-4127-42a6-b23c-86d89534305b-istio-podinfo\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.365863 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.365836    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-workload-certs\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.365974 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.365845    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-credential-socket\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.365974 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.365890    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f26433d-4127-42a6-b23c-86d89534305b-istiod-ca-cert\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.365974 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.365890    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-workload-socket\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.367828 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.367809    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-istio-envoy\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.368171 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.368151    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f26433d-4127-42a6-b23c-86d89534305b-istio-podinfo\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.378459 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.378433    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f26433d-4127-42a6-b23c-86d89534305b-istio-token\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.378673 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.378657    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmppn\" (UniqueName: \"kubernetes.io/projected/4f26433d-4127-42a6-b23c-86d89534305b-kube-api-access-zmppn\") pod \"e2e-api-a-40c586-openshift-default-68c58bff8d-phz44\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") " pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.485949 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.485920    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:10.763401 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.763374    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"]
Sep 14 03:15:10.763763 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:15:10.763745    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f26433d_4127_42a6_b23c_86d89534305b.slice/crio-5b0521510215921550f10d1febe1117f9790ff25e8f569f1dec28e258e8170e5 WatchSource:0}: Error finding container 5b0521510215921550f10d1febe1117f9790ff25e8f569f1dec28e258e8170e5: Status 404 returned error can't find the container with id 5b0521510215921550f10d1febe1117f9790ff25e8f569f1dec28e258e8170e5
Sep 14 03:15:10.765890 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.765859    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:10.765958 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.765921    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:10.765958 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:10.765949    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:11.558837 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:11.558800    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44" event={"ID":"4f26433d-4127-42a6-b23c-86d89534305b","Type":"ContainerStarted","Data":"4c173396c90e09a0ec4d5a765323353b3d8a041c991cf0ad67b54b5f56136e6d"}
Sep 14 03:15:11.559188 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:11.558869    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44" event={"ID":"4f26433d-4127-42a6-b23c-86d89534305b","Type":"ContainerStarted","Data":"5b0521510215921550f10d1febe1117f9790ff25e8f569f1dec28e258e8170e5"}
Sep 14 03:15:11.603195 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:11.603149    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44" podStartSLOduration=1.603132878 podStartE2EDuration="1.603132878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:11.601376479 +0000 UTC m=+1457.914400298" watchObservedRunningTime="2026-09-14 03:15:11.603132878 +0000 UTC m=+1457.916156696"
Sep 14 03:15:12.486154 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:12.486111    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:12.490744 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:12.490723    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:12.563905 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:12.563879    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:12.564621 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:12.564604    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:16.107629 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.107586    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8" podUID="1277df0a-0d9d-4cd5-9530-70f23c8438a5" containerName="main" containerID="cri-o://775057f37ec1dcd1652af1b1732784805a1fc71b4cc3ac42649d71b76218460c" gracePeriod=45
Sep 14 03:15:16.351098 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.351074    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:15:16.528336 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.528305    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1277df0a-0d9d-4cd5-9530-70f23c8438a5-tls-certs\") pod \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") "
Sep 14 03:15:16.528548 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.528423    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-home\") pod \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") "
Sep 14 03:15:16.528548 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.528470    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-model-cache\") pod \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") "
Sep 14 03:15:16.528548 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.528498    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-dshm\") pod \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") "
Sep 14 03:15:16.528732 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.528569    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9png\" (UniqueName: \"kubernetes.io/projected/1277df0a-0d9d-4cd5-9530-70f23c8438a5-kube-api-access-p9png\") pod \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") "
Sep 14 03:15:16.528732 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.528610    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-tmp-dir\") pod \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\" (UID: \"1277df0a-0d9d-4cd5-9530-70f23c8438a5\") "
Sep 14 03:15:16.528834 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.528739    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-home" (OuterVolumeSpecName: "home") pod "1277df0a-0d9d-4cd5-9530-70f23c8438a5" (UID: "1277df0a-0d9d-4cd5-9530-70f23c8438a5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:15:16.528834 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.528749    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-model-cache" (OuterVolumeSpecName: "model-cache") pod "1277df0a-0d9d-4cd5-9530-70f23c8438a5" (UID: "1277df0a-0d9d-4cd5-9530-70f23c8438a5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:15:16.528951 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.528931    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:15:16.529008 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.528956    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:15:16.529008 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.528929    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1277df0a-0d9d-4cd5-9530-70f23c8438a5" (UID: "1277df0a-0d9d-4cd5-9530-70f23c8438a5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:15:16.530767 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.530747    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-dshm" (OuterVolumeSpecName: "dshm") pod "1277df0a-0d9d-4cd5-9530-70f23c8438a5" (UID: "1277df0a-0d9d-4cd5-9530-70f23c8438a5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:15:16.531190 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.531168    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1277df0a-0d9d-4cd5-9530-70f23c8438a5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1277df0a-0d9d-4cd5-9530-70f23c8438a5" (UID: "1277df0a-0d9d-4cd5-9530-70f23c8438a5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:15:16.531190 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.531181    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1277df0a-0d9d-4cd5-9530-70f23c8438a5-kube-api-access-p9png" (OuterVolumeSpecName: "kube-api-access-p9png") pod "1277df0a-0d9d-4cd5-9530-70f23c8438a5" (UID: "1277df0a-0d9d-4cd5-9530-70f23c8438a5"). InnerVolumeSpecName "kube-api-access-p9png". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:15:16.589861 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.586881    2574 generic.go:356] "Generic (PLEG): container finished" podID="1277df0a-0d9d-4cd5-9530-70f23c8438a5" containerID="775057f37ec1dcd1652af1b1732784805a1fc71b4cc3ac42649d71b76218460c" exitCode=0
Sep 14 03:15:16.589861 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.586975    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8" event={"ID":"1277df0a-0d9d-4cd5-9530-70f23c8438a5","Type":"ContainerDied","Data":"775057f37ec1dcd1652af1b1732784805a1fc71b4cc3ac42649d71b76218460c"}
Sep 14 03:15:16.589861 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.587013    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8" event={"ID":"1277df0a-0d9d-4cd5-9530-70f23c8438a5","Type":"ContainerDied","Data":"4427d9d1d62faf751867807a5991a322867e926f351f69a109cab0c83469c646"}
Sep 14 03:15:16.589861 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.587037    2574 scope.go:117] "RemoveContainer" containerID="775057f37ec1dcd1652af1b1732784805a1fc71b4cc3ac42649d71b76218460c"
Sep 14 03:15:16.589861 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.587218    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"
Sep 14 03:15:16.599852 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.599833    2574 scope.go:117] "RemoveContainer" containerID="775057f37ec1dcd1652af1b1732784805a1fc71b4cc3ac42649d71b76218460c"
Sep 14 03:15:16.600110 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:15:16.600093    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"775057f37ec1dcd1652af1b1732784805a1fc71b4cc3ac42649d71b76218460c\": container with ID starting with 775057f37ec1dcd1652af1b1732784805a1fc71b4cc3ac42649d71b76218460c not found: ID does not exist" containerID="775057f37ec1dcd1652af1b1732784805a1fc71b4cc3ac42649d71b76218460c"
Sep 14 03:15:16.600179 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.600117    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"775057f37ec1dcd1652af1b1732784805a1fc71b4cc3ac42649d71b76218460c"} err="failed to get container status \"775057f37ec1dcd1652af1b1732784805a1fc71b4cc3ac42649d71b76218460c\": rpc error: code = NotFound desc = could not find container \"775057f37ec1dcd1652af1b1732784805a1fc71b4cc3ac42649d71b76218460c\": container with ID starting with 775057f37ec1dcd1652af1b1732784805a1fc71b4cc3ac42649d71b76218460c not found: ID does not exist"
Sep 14 03:15:16.611656 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.611629    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"]
Sep 14 03:15:16.617816 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.617795    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-0c919e/rate-test-model-0c919e-kserve-77f4794c79-nn5c8"]
Sep 14 03:15:16.630266 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.630245    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1277df0a-0d9d-4cd5-9530-70f23c8438a5-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:15:16.630266 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.630263    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:15:16.630360 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.630274    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9png\" (UniqueName: \"kubernetes.io/projected/1277df0a-0d9d-4cd5-9530-70f23c8438a5-kube-api-access-p9png\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:15:16.630360 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:16.630283    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1277df0a-0d9d-4cd5-9530-70f23c8438a5-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:15:17.105679 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.105639    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br" podUID="7535bc6b-9712-412a-a91c-3013cd07b520" containerName="main" containerID="cri-o://12bb374e4c911dcfb0287e0f9e9a890313096c618c33396df63fe56e1e1ecf78" gracePeriod=45
Sep 14 03:15:17.399621 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.399587    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:15:17.538146 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.538112    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-home\") pod \"7535bc6b-9712-412a-a91c-3013cd07b520\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") "
Sep 14 03:15:17.538317 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.538166    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-model-cache\") pod \"7535bc6b-9712-412a-a91c-3013cd07b520\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") "
Sep 14 03:15:17.538317 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.538227    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-dshm\") pod \"7535bc6b-9712-412a-a91c-3013cd07b520\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") "
Sep 14 03:15:17.538317 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.538267    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7535bc6b-9712-412a-a91c-3013cd07b520-tls-certs\") pod \"7535bc6b-9712-412a-a91c-3013cd07b520\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") "
Sep 14 03:15:17.538317 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.538302    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkr5r\" (UniqueName: \"kubernetes.io/projected/7535bc6b-9712-412a-a91c-3013cd07b520-kube-api-access-nkr5r\") pod \"7535bc6b-9712-412a-a91c-3013cd07b520\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") "
Sep 14 03:15:17.538500 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.538330    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-tmp-dir\") pod \"7535bc6b-9712-412a-a91c-3013cd07b520\" (UID: \"7535bc6b-9712-412a-a91c-3013cd07b520\") "
Sep 14 03:15:17.538500 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.538377    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-home" (OuterVolumeSpecName: "home") pod "7535bc6b-9712-412a-a91c-3013cd07b520" (UID: "7535bc6b-9712-412a-a91c-3013cd07b520"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:15:17.538500 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.538449    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-model-cache" (OuterVolumeSpecName: "model-cache") pod "7535bc6b-9712-412a-a91c-3013cd07b520" (UID: "7535bc6b-9712-412a-a91c-3013cd07b520"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:15:17.538709 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.538652    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7535bc6b-9712-412a-a91c-3013cd07b520" (UID: "7535bc6b-9712-412a-a91c-3013cd07b520"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:15:17.538768 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.538756    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:15:17.538830 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.538776    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:15:17.538830 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.538801    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:15:17.540603 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.540572    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-dshm" (OuterVolumeSpecName: "dshm") pod "7535bc6b-9712-412a-a91c-3013cd07b520" (UID: "7535bc6b-9712-412a-a91c-3013cd07b520"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:15:17.540719 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.540622    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7535bc6b-9712-412a-a91c-3013cd07b520-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7535bc6b-9712-412a-a91c-3013cd07b520" (UID: "7535bc6b-9712-412a-a91c-3013cd07b520"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:15:17.540719 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.540634    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7535bc6b-9712-412a-a91c-3013cd07b520-kube-api-access-nkr5r" (OuterVolumeSpecName: "kube-api-access-nkr5r") pod "7535bc6b-9712-412a-a91c-3013cd07b520" (UID: "7535bc6b-9712-412a-a91c-3013cd07b520"). InnerVolumeSpecName "kube-api-access-nkr5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:15:17.595057 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.595021    2574 generic.go:356] "Generic (PLEG): container finished" podID="7535bc6b-9712-412a-a91c-3013cd07b520" containerID="12bb374e4c911dcfb0287e0f9e9a890313096c618c33396df63fe56e1e1ecf78" exitCode=0
Sep 14 03:15:17.595220 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.595060    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"
Sep 14 03:15:17.595220 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.595111    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br" event={"ID":"7535bc6b-9712-412a-a91c-3013cd07b520","Type":"ContainerDied","Data":"12bb374e4c911dcfb0287e0f9e9a890313096c618c33396df63fe56e1e1ecf78"}
Sep 14 03:15:17.595220 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.595153    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br" event={"ID":"7535bc6b-9712-412a-a91c-3013cd07b520","Type":"ContainerDied","Data":"f0fc00e5f0ad0a4399555a35488456728665c31752b337c5023bfd87fd7403f3"}
Sep 14 03:15:17.595220 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.595175    2574 scope.go:117] "RemoveContainer" containerID="12bb374e4c911dcfb0287e0f9e9a890313096c618c33396df63fe56e1e1ecf78"
Sep 14 03:15:17.604682 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.604660    2574 scope.go:117] "RemoveContainer" containerID="12bb374e4c911dcfb0287e0f9e9a890313096c618c33396df63fe56e1e1ecf78"
Sep 14 03:15:17.604926 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:15:17.604909    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12bb374e4c911dcfb0287e0f9e9a890313096c618c33396df63fe56e1e1ecf78\": container with ID starting with 12bb374e4c911dcfb0287e0f9e9a890313096c618c33396df63fe56e1e1ecf78 not found: ID does not exist" containerID="12bb374e4c911dcfb0287e0f9e9a890313096c618c33396df63fe56e1e1ecf78"
Sep 14 03:15:17.604975 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.604934    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12bb374e4c911dcfb0287e0f9e9a890313096c618c33396df63fe56e1e1ecf78"} err="failed to get container status \"12bb374e4c911dcfb0287e0f9e9a890313096c618c33396df63fe56e1e1ecf78\": rpc error: code = NotFound desc = could not find container \"12bb374e4c911dcfb0287e0f9e9a890313096c618c33396df63fe56e1e1ecf78\": container with ID starting with 12bb374e4c911dcfb0287e0f9e9a890313096c618c33396df63fe56e1e1ecf78 not found: ID does not exist"
Sep 14 03:15:17.618770 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.618714    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"]
Sep 14 03:15:17.622422 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.622403    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-f83c87/rate-test-model-f83c87-kserve-7579f9887-dm9br"]
Sep 14 03:15:17.639926 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.639905    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkr5r\" (UniqueName: \"kubernetes.io/projected/7535bc6b-9712-412a-a91c-3013cd07b520-kube-api-access-nkr5r\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:15:17.639926 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.639925    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7535bc6b-9712-412a-a91c-3013cd07b520-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:15:17.640101 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.639935    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7535bc6b-9712-412a-a91c-3013cd07b520-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:15:17.935929 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.935837    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"]
Sep 14 03:15:17.936259 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.936244    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7535bc6b-9712-412a-a91c-3013cd07b520" containerName="main"
Sep 14 03:15:17.936329 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.936260    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7535bc6b-9712-412a-a91c-3013cd07b520" containerName="main"
Sep 14 03:15:17.936329 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.936272    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1277df0a-0d9d-4cd5-9530-70f23c8438a5" containerName="main"
Sep 14 03:15:17.936329 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.936278    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1277df0a-0d9d-4cd5-9530-70f23c8438a5" containerName="main"
Sep 14 03:15:17.936492 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.936338    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1277df0a-0d9d-4cd5-9530-70f23c8438a5" containerName="main"
Sep 14 03:15:17.936492 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.936348    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7535bc6b-9712-412a-a91c-3013cd07b520" containerName="main"
Sep 14 03:15:17.940704 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.940688    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"
Sep 14 03:15:17.943428 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.943407    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-40c586\""
Sep 14 03:15:17.948606 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:17.948587    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"]
Sep 14 03:15:18.043161 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:18.043130    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtqwm\" (UniqueName: \"kubernetes.io/projected/662dade8-9aa2-4025-8ae2-a694448e4f3d-kube-api-access-dtqwm\") pod \"payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7\" (UID: \"662dade8-9aa2-4025-8ae2-a694448e4f3d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"
Sep 14 03:15:18.043306 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:18.043209    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/662dade8-9aa2-4025-8ae2-a694448e4f3d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7\" (UID: \"662dade8-9aa2-4025-8ae2-a694448e4f3d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"
Sep 14 03:15:18.144574 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:18.144537    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtqwm\" (UniqueName: \"kubernetes.io/projected/662dade8-9aa2-4025-8ae2-a694448e4f3d-kube-api-access-dtqwm\") pod \"payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7\" (UID: \"662dade8-9aa2-4025-8ae2-a694448e4f3d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"
Sep 14 03:15:18.144743 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:18.144616    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/662dade8-9aa2-4025-8ae2-a694448e4f3d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7\" (UID: \"662dade8-9aa2-4025-8ae2-a694448e4f3d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"
Sep 14 03:15:18.145272 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:18.145254    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/662dade8-9aa2-4025-8ae2-a694448e4f3d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7\" (UID: \"662dade8-9aa2-4025-8ae2-a694448e4f3d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"
Sep 14 03:15:18.153075 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:18.153053    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtqwm\" (UniqueName: \"kubernetes.io/projected/662dade8-9aa2-4025-8ae2-a694448e4f3d-kube-api-access-dtqwm\") pod \"payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7\" (UID: \"662dade8-9aa2-4025-8ae2-a694448e4f3d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"
Sep 14 03:15:18.219151 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:18.219066    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1277df0a-0d9d-4cd5-9530-70f23c8438a5" path="/var/lib/kubelet/pods/1277df0a-0d9d-4cd5-9530-70f23c8438a5/volumes"
Sep 14 03:15:18.219602 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:18.219579    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7535bc6b-9712-412a-a91c-3013cd07b520" path="/var/lib/kubelet/pods/7535bc6b-9712-412a-a91c-3013cd07b520/volumes"
Sep 14 03:15:18.251769 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:18.251746    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"
Sep 14 03:15:18.548289 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:18.548263    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"]
Sep 14 03:15:18.549684 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:15:18.549651    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod662dade8_9aa2_4025_8ae2_a694448e4f3d.slice/crio-babd5e29c506a1fb041536f6398686e7c6ba0b2b9ebb14086fab3f20df2f6f71 WatchSource:0}: Error finding container babd5e29c506a1fb041536f6398686e7c6ba0b2b9ebb14086fab3f20df2f6f71: Status 404 returned error can't find the container with id babd5e29c506a1fb041536f6398686e7c6ba0b2b9ebb14086fab3f20df2f6f71
Sep 14 03:15:18.551588 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:18.551568    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 03:15:18.602209 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:18.602180    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7" event={"ID":"662dade8-9aa2-4025-8ae2-a694448e4f3d","Type":"ContainerStarted","Data":"babd5e29c506a1fb041536f6398686e7c6ba0b2b9ebb14086fab3f20df2f6f71"}
Sep 14 03:15:20.615551 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:20.615485    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7" event={"ID":"662dade8-9aa2-4025-8ae2-a694448e4f3d","Type":"ContainerStarted","Data":"0f354cc1e10dbc94f311dff4e9763fe916784d1acce306ab7f70d9bc1d35a613"}
Sep 14 03:15:20.616013 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:20.615589    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"
Sep 14 03:15:20.632089 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:20.632041    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7" podStartSLOduration=2.179553485 podStartE2EDuration="3.632025978s" totalImagesPullingTime="1.452472493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:20.631825039 +0000 UTC m=+1466.944848861" watchObservedRunningTime="2026-09-14 03:15:20.632025978 +0000 UTC m=+1466.945049825"
Sep 14 03:15:23.125692 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:23.125653    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:15:25.644996 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:25.644969    2574 generic.go:356] "Generic (PLEG): container finished" podID="a5159351-6b1d-4ddd-a385-e7ea69d07591" containerID="92b132094e3289191e00761fe19caf53a48426635013d37f306b610946c025ae" exitCode=0
Sep 14 03:15:25.645362 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:25.645037    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z" event={"ID":"a5159351-6b1d-4ddd-a385-e7ea69d07591","Type":"ContainerDied","Data":"92b132094e3289191e00761fe19caf53a48426635013d37f306b610946c025ae"}
Sep 14 03:15:25.645441 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:25.645426    2574 scope.go:117] "RemoveContainer" containerID="92b132094e3289191e00761fe19caf53a48426635013d37f306b610946c025ae"
Sep 14 03:15:25.646580 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:25.646556    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:25.646648 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:25.646622    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:25.646687 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:25.646669    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:26.652848 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:26.652785    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z" event={"ID":"a5159351-6b1d-4ddd-a385-e7ea69d07591","Type":"ContainerStarted","Data":"06fa898d26da54eaa2af0d28828722b825d0cb80e650665ed1eb17c3d08860c8"}
Sep 14 03:15:27.185186 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:27.185152    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:27.185595 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:27.185490    2574 patch_prober.go:28] interesting pod/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 14 03:15:27.185784 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:27.185759    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z" podUID="a5159351-6b1d-4ddd-a385-e7ea69d07591" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 14 03:15:28.185277 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:28.185235    2574 patch_prober.go:28] interesting pod/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 14 03:15:28.185912 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:28.185879    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z" podUID="a5159351-6b1d-4ddd-a385-e7ea69d07591" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 14 03:15:29.184786 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:29.184748    2574 patch_prober.go:28] interesting pod/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 14 03:15:29.185081 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:29.185049    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z" podUID="a5159351-6b1d-4ddd-a385-e7ea69d07591" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 14 03:15:30.184313 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:30.184274    2574 patch_prober.go:28] interesting pod/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 14 03:15:30.184944 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:30.184910    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z" podUID="a5159351-6b1d-4ddd-a385-e7ea69d07591" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 14 03:15:31.186015 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:31.185551    2574 patch_prober.go:28] interesting pod/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 14 03:15:31.186015 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:31.185617    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z" podUID="a5159351-6b1d-4ddd-a385-e7ea69d07591" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 14 03:15:31.623493 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:31.623456    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"
Sep 14 03:15:32.191640 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:32.191610    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:32.192126 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:32.191973    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:32.193480 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:32.193456    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:15:34.717408 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:34.717374    2574 generic.go:356] "Generic (PLEG): container finished" podID="4f26433d-4127-42a6-b23c-86d89534305b" containerID="4c173396c90e09a0ec4d5a765323353b3d8a041c991cf0ad67b54b5f56136e6d" exitCode=0
Sep 14 03:15:34.717862 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:34.717468    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44" event={"ID":"4f26433d-4127-42a6-b23c-86d89534305b","Type":"ContainerDied","Data":"4c173396c90e09a0ec4d5a765323353b3d8a041c991cf0ad67b54b5f56136e6d"}
Sep 14 03:15:34.717905 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:34.717886    2574 scope.go:117] "RemoveContainer" containerID="4c173396c90e09a0ec4d5a765323353b3d8a041c991cf0ad67b54b5f56136e6d"
Sep 14 03:15:34.719137 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:34.719112    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:34.719223 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:34.719186    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:34.719223 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:34.719218    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:35.724435 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:35.724397    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44" event={"ID":"4f26433d-4127-42a6-b23c-86d89534305b","Type":"ContainerStarted","Data":"5e7751818234abf696628d1d1a7da4531e10bbdc08fee0ea823e3bc7f2a4372f"}
Sep 14 03:15:36.486607 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:36.486567    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:36.487101 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:36.487072    2574 patch_prober.go:28] interesting pod/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 14 03:15:36.487347 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:36.487305    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44" podUID="4f26433d-4127-42a6-b23c-86d89534305b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 14 03:15:37.485538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.485490    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"]
Sep 14 03:15:37.492567 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.486353    2574 patch_prober.go:28] interesting pod/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 14 03:15:37.492790 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.492761    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44" podUID="4f26433d-4127-42a6-b23c-86d89534305b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 14 03:15:37.497705 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.497677    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.508054 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.508037    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-bdf5b3-openshift-default-dockercfg-vj6vb\""
Sep 14 03:15:37.523639 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.523616    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"]
Sep 14 03:15:37.544546 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.544504    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/752c7a53-2483-43b6-a723-30e32618e40c-istio-token\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.544644 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.544565    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-istio-data\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.544710 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.544657    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/752c7a53-2483-43b6-a723-30e32618e40c-istio-podinfo\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.544710 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.544692    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/752c7a53-2483-43b6-a723-30e32618e40c-istiod-ca-cert\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.544802 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.544717    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h9kp\" (UniqueName: \"kubernetes.io/projected/752c7a53-2483-43b6-a723-30e32618e40c-kube-api-access-9h9kp\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.544802 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.544746    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-workload-certs\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.544902 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.544800    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-credential-socket\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.544902 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.544866    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-workload-socket\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.544991 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.544905    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-istio-envoy\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.645561 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.645494    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/752c7a53-2483-43b6-a723-30e32618e40c-istio-podinfo\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.645762 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.645565    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/752c7a53-2483-43b6-a723-30e32618e40c-istiod-ca-cert\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.645762 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.645591    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9kp\" (UniqueName: \"kubernetes.io/projected/752c7a53-2483-43b6-a723-30e32618e40c-kube-api-access-9h9kp\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.645762 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.645615    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-workload-certs\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.645762 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.645648    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-credential-socket\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.645762 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.645713    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-workload-socket\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.646029 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.645766    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-istio-envoy\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.646029 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.645817    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/752c7a53-2483-43b6-a723-30e32618e40c-istio-token\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.646029 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.645843    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-istio-data\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.646198 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.646082    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-workload-certs\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.646198 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.646135    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-credential-socket\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.646300 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.646246    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-istio-data\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.646300 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.646293    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/752c7a53-2483-43b6-a723-30e32618e40c-istiod-ca-cert\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.646387 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.646366    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-workload-socket\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.648270 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.648241    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-istio-envoy\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.648377 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.648364    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/752c7a53-2483-43b6-a723-30e32618e40c-istio-podinfo\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.657015 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.656997    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/752c7a53-2483-43b6-a723-30e32618e40c-istio-token\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.664576 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.664552    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h9kp\" (UniqueName: \"kubernetes.io/projected/752c7a53-2483-43b6-a723-30e32618e40c-kube-api-access-9h9kp\") pod \"e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") " pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:37.808556 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:37.808437    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:38.093958 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:38.093935    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"]
Sep 14 03:15:38.096095 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:15:38.096069    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod752c7a53_2483_43b6_a723_30e32618e40c.slice/crio-2e4d4818ba075d28112cceea20da327f7b2901df45c3c4252d524ab8fcb6f4ba WatchSource:0}: Error finding container 2e4d4818ba075d28112cceea20da327f7b2901df45c3c4252d524ab8fcb6f4ba: Status 404 returned error can't find the container with id 2e4d4818ba075d28112cceea20da327f7b2901df45c3c4252d524ab8fcb6f4ba
Sep 14 03:15:38.098797 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:38.098752    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:38.098896 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:38.098831    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:38.098896 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:38.098867    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:38.486443 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:38.486409    2574 patch_prober.go:28] interesting pod/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 14 03:15:38.486941 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:38.486478    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44" podUID="4f26433d-4127-42a6-b23c-86d89534305b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 14 03:15:38.742273 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:38.742188    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s" event={"ID":"752c7a53-2483-43b6-a723-30e32618e40c","Type":"ContainerStarted","Data":"cceff37ea00daba6c65dfb1e192feb049211425b5fa8bc97713bef38a0b00514"}
Sep 14 03:15:38.742273 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:38.742239    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s" event={"ID":"752c7a53-2483-43b6-a723-30e32618e40c","Type":"ContainerStarted","Data":"2e4d4818ba075d28112cceea20da327f7b2901df45c3c4252d524ab8fcb6f4ba"}
Sep 14 03:15:38.766220 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:38.766149    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s" podStartSLOduration=1.766131651 podStartE2EDuration="1.766131651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:38.764028329 +0000 UTC m=+1485.077052149" watchObservedRunningTime="2026-09-14 03:15:38.766131651 +0000 UTC m=+1485.079155471"
Sep 14 03:15:38.808976 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:38.808953    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:39.227920 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.227895    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"]
Sep 14 03:15:39.232078 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.232056    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.236029 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.236009    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-13d31c-openshift-default-dockercfg-ms9lm\""
Sep 14 03:15:39.247955 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.247927    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"]
Sep 14 03:15:39.262183 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.262159    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-istio-data\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.262280 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.262194    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/974716b6-453d-4626-8ce9-727784c1e8a5-istio-podinfo\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.262280 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.262217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/974716b6-453d-4626-8ce9-727784c1e8a5-istio-token\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.262353 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.262303    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-workload-certs\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.262353 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.262336    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-istio-envoy\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.262419 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.262365    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz9l2\" (UniqueName: \"kubernetes.io/projected/974716b6-453d-4626-8ce9-727784c1e8a5-kube-api-access-bz9l2\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.262419 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.262404    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-credential-socket\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.262480 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.262430    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/974716b6-453d-4626-8ce9-727784c1e8a5-istiod-ca-cert\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.262480 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.262469    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-workload-socket\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.363928 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.363885    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-workload-certs\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.363928 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.363924    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-istio-envoy\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.364193 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.363943    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz9l2\" (UniqueName: \"kubernetes.io/projected/974716b6-453d-4626-8ce9-727784c1e8a5-kube-api-access-bz9l2\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.364193 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.363979    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-credential-socket\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.364193 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.363998    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/974716b6-453d-4626-8ce9-727784c1e8a5-istiod-ca-cert\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.364193 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.364035    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-workload-socket\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.364193 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.364074    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-istio-data\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.364193 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.364097    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/974716b6-453d-4626-8ce9-727784c1e8a5-istio-podinfo\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.364193 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.364120    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/974716b6-453d-4626-8ce9-727784c1e8a5-istio-token\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.364582 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.364365    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-workload-certs\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.364582 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.364540    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-workload-socket\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.364673 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.364577    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-istio-data\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.364731 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.364711    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/974716b6-453d-4626-8ce9-727784c1e8a5-istiod-ca-cert\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.364819 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.364800    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-credential-socket\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.369761 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.369738    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-istio-envoy\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.370401 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.370350    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/974716b6-453d-4626-8ce9-727784c1e8a5-istio-podinfo\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.380021 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.379998    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/974716b6-453d-4626-8ce9-727784c1e8a5-istio-token\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.380808 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.380785    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz9l2\" (UniqueName: \"kubernetes.io/projected/974716b6-453d-4626-8ce9-727784c1e8a5-kube-api-access-bz9l2\") pod \"e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") " pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.487417 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.487328    2574 patch_prober.go:28] interesting pod/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 14 03:15:39.488001 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.487968    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44" podUID="4f26433d-4127-42a6-b23c-86d89534305b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 14 03:15:39.543815 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.543794    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:39.813822 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.813749    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:39.847784 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.847759    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"]
Sep 14 03:15:39.849769 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:15:39.849746    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod974716b6_453d_4626_8ce9_727784c1e8a5.slice/crio-41abdb0b4e1b3a6b68aafe128e388dc24dbe27f82e116f440e9102b1944a1355 WatchSource:0}: Error finding container 41abdb0b4e1b3a6b68aafe128e388dc24dbe27f82e116f440e9102b1944a1355: Status 404 returned error can't find the container with id 41abdb0b4e1b3a6b68aafe128e388dc24dbe27f82e116f440e9102b1944a1355
Sep 14 03:15:39.852150 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.852121    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:39.852229 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.852184    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:39.852269 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:39.852233    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:15:40.486894 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:40.486852    2574 patch_prober.go:28] interesting pod/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 14 03:15:40.487105 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:40.486916    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44" podUID="4f26433d-4127-42a6-b23c-86d89534305b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 14 03:15:40.755466 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:40.755379    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb" event={"ID":"974716b6-453d-4626-8ce9-727784c1e8a5","Type":"ContainerStarted","Data":"55c3e4d713c0457471f84c91391753927c9721b3cd7e46c738317926a5e1ead0"}
Sep 14 03:15:40.755466 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:40.755435    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb" event={"ID":"974716b6-453d-4626-8ce9-727784c1e8a5","Type":"ContainerStarted","Data":"41abdb0b4e1b3a6b68aafe128e388dc24dbe27f82e116f440e9102b1944a1355"}
Sep 14 03:15:40.755984 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:40.755734    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:40.756618 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:40.756602    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:15:40.786920 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:40.786862    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb" podStartSLOduration=1.786846272 podStartE2EDuration="1.786846272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:40.785662551 +0000 UTC m=+1487.098686407" watchObservedRunningTime="2026-09-14 03:15:40.786846272 +0000 UTC m=+1487.099870091"
Sep 14 03:15:41.496689 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:41.496654    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:41.496959 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:41.496923    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:41.497603 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:41.497586    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:15:41.544698 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:41.544676    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:41.551586 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:41.551560    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:41.761292 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:41.761193    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:41.762177 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:41.762160    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:15:44.827304 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:44.827271    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"]
Sep 14 03:15:44.830864 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:44.830847    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"
Sep 14 03:15:44.833654 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:44.833629    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-bdf5b3\""
Sep 14 03:15:44.833757 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:44.833670    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-bdf5b3-dockercfg-vsn4g\""
Sep 14 03:15:44.842120 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:44.842098    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"]
Sep 14 03:15:44.855265 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:44.855240    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"]
Sep 14 03:15:44.858629 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:44.858610    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"
Sep 14 03:15:44.870890 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:44.870871    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"]
Sep 14 03:15:44.935118 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:44.935078    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/347cad42-21dd-426e-9694-24a16932e9d7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz\" (UID: \"347cad42-21dd-426e-9694-24a16932e9d7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"
Sep 14 03:15:44.935433 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:44.935408    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf16ad4-b620-4fa6-a95f-9a899f6e58dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4\" (UID: \"fcf16ad4-b620-4fa6-a95f-9a899f6e58dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"
Sep 14 03:15:44.935617 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:44.935599    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gnmn\" (UniqueName: \"kubernetes.io/projected/347cad42-21dd-426e-9694-24a16932e9d7-kube-api-access-2gnmn\") pod \"payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz\" (UID: \"347cad42-21dd-426e-9694-24a16932e9d7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"
Sep 14 03:15:44.935766 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:44.935751    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csb2s\" (UniqueName: \"kubernetes.io/projected/fcf16ad4-b620-4fa6-a95f-9a899f6e58dd-kube-api-access-csb2s\") pod \"payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4\" (UID: \"fcf16ad4-b620-4fa6-a95f-9a899f6e58dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"
Sep 14 03:15:45.037442 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.037405    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/347cad42-21dd-426e-9694-24a16932e9d7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz\" (UID: \"347cad42-21dd-426e-9694-24a16932e9d7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"
Sep 14 03:15:45.037442 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.037444    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf16ad4-b620-4fa6-a95f-9a899f6e58dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4\" (UID: \"fcf16ad4-b620-4fa6-a95f-9a899f6e58dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"
Sep 14 03:15:45.037735 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.037638    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gnmn\" (UniqueName: \"kubernetes.io/projected/347cad42-21dd-426e-9694-24a16932e9d7-kube-api-access-2gnmn\") pod \"payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz\" (UID: \"347cad42-21dd-426e-9694-24a16932e9d7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"
Sep 14 03:15:45.037735 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.037668    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csb2s\" (UniqueName: \"kubernetes.io/projected/fcf16ad4-b620-4fa6-a95f-9a899f6e58dd-kube-api-access-csb2s\") pod \"payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4\" (UID: \"fcf16ad4-b620-4fa6-a95f-9a899f6e58dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"
Sep 14 03:15:45.038245 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.038221    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/347cad42-21dd-426e-9694-24a16932e9d7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz\" (UID: \"347cad42-21dd-426e-9694-24a16932e9d7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"
Sep 14 03:15:45.038336 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.038270    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf16ad4-b620-4fa6-a95f-9a899f6e58dd-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4\" (UID: \"fcf16ad4-b620-4fa6-a95f-9a899f6e58dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"
Sep 14 03:15:45.046286 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.046258    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gnmn\" (UniqueName: \"kubernetes.io/projected/347cad42-21dd-426e-9694-24a16932e9d7-kube-api-access-2gnmn\") pod \"payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz\" (UID: \"347cad42-21dd-426e-9694-24a16932e9d7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"
Sep 14 03:15:45.046398 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.046320    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csb2s\" (UniqueName: \"kubernetes.io/projected/fcf16ad4-b620-4fa6-a95f-9a899f6e58dd-kube-api-access-csb2s\") pod \"payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4\" (UID: \"fcf16ad4-b620-4fa6-a95f-9a899f6e58dd\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"
Sep 14 03:15:45.141958 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.141893    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"
Sep 14 03:15:45.172199 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.172108    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"
Sep 14 03:15:45.420926 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.420901    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"]
Sep 14 03:15:45.422808 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:15:45.422768    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf16ad4_b620_4fa6_a95f_9a899f6e58dd.slice/crio-98baba3e110d99e812e2785259dd1697714ed4c48734c12caadfe96b2001b5c7 WatchSource:0}: Error finding container 98baba3e110d99e812e2785259dd1697714ed4c48734c12caadfe96b2001b5c7: Status 404 returned error can't find the container with id 98baba3e110d99e812e2785259dd1697714ed4c48734c12caadfe96b2001b5c7
Sep 14 03:15:45.460362 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.460328    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"]
Sep 14 03:15:45.461783 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:15:45.461761    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod347cad42_21dd_426e_9694_24a16932e9d7.slice/crio-0aab68c6f8933c77d7c173010cca5f922f427a37b43a0699233610a910da71a1 WatchSource:0}: Error finding container 0aab68c6f8933c77d7c173010cca5f922f427a37b43a0699233610a910da71a1: Status 404 returned error can't find the container with id 0aab68c6f8933c77d7c173010cca5f922f427a37b43a0699233610a910da71a1
Sep 14 03:15:45.786281 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.786246    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4" event={"ID":"fcf16ad4-b620-4fa6-a95f-9a899f6e58dd","Type":"ContainerStarted","Data":"35e56e3c870bdc4464c07d2b21341a25740cac075a9b2d97f83296eb1cbecbfd"}
Sep 14 03:15:45.786488 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.786315    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"
Sep 14 03:15:45.786488 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.786341    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4" event={"ID":"fcf16ad4-b620-4fa6-a95f-9a899f6e58dd","Type":"ContainerStarted","Data":"98baba3e110d99e812e2785259dd1697714ed4c48734c12caadfe96b2001b5c7"}
Sep 14 03:15:45.788234 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.788201    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz" event={"ID":"347cad42-21dd-426e-9694-24a16932e9d7","Type":"ContainerStarted","Data":"764b31c0b769a9337343a71ec82756e4e76ea5bfce04552e5b5bb3818ddaeba7"}
Sep 14 03:15:45.788335 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.788246    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz" event={"ID":"347cad42-21dd-426e-9694-24a16932e9d7","Type":"ContainerStarted","Data":"0aab68c6f8933c77d7c173010cca5f922f427a37b43a0699233610a910da71a1"}
Sep 14 03:15:45.788392 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.788330    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"
Sep 14 03:15:45.811116 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:45.811070    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4" podStartSLOduration=1.811056065 podStartE2EDuration="1.811056065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:45.809887082 +0000 UTC m=+1492.122910901" watchObservedRunningTime="2026-09-14 03:15:45.811056065 +0000 UTC m=+1492.124079884"
Sep 14 03:15:51.040481 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:51.040423    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz" podStartSLOduration=7.040406333 podStartE2EDuration="7.040406333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:45.834710064 +0000 UTC m=+1492.147733887" watchObservedRunningTime="2026-09-14 03:15:51.040406333 +0000 UTC m=+1497.353430152"
Sep 14 03:15:51.040905 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:51.040890    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:15:54.220177 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:54.220151    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 03:15:54.226630 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:54.226606    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 03:15:54.258855 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:54.258835    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 03:15:54.264946 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:54.264925    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 03:15:54.410597 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:54.410571    2574 scope.go:117] "RemoveContainer" containerID="fab86dc33f54006c453e046ee67204def360c0561a27c3d663b1df15a11c7da9"
Sep 14 03:15:56.796454 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:56.796428    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"
Sep 14 03:15:56.796845 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:15:56.796560    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"
Sep 14 03:16:01.893235 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:01.893204    2574 generic.go:356] "Generic (PLEG): container finished" podID="752c7a53-2483-43b6-a723-30e32618e40c" containerID="cceff37ea00daba6c65dfb1e192feb049211425b5fa8bc97713bef38a0b00514" exitCode=0
Sep 14 03:16:01.893643 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:01.893285    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s" event={"ID":"752c7a53-2483-43b6-a723-30e32618e40c","Type":"ContainerDied","Data":"cceff37ea00daba6c65dfb1e192feb049211425b5fa8bc97713bef38a0b00514"}
Sep 14 03:16:01.893853 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:01.893834    2574 scope.go:117] "RemoveContainer" containerID="cceff37ea00daba6c65dfb1e192feb049211425b5fa8bc97713bef38a0b00514"
Sep 14 03:16:01.895213 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:01.895184    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:16:01.895286 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:01.895253    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:16:01.895325 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:01.895282    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:16:02.900276 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:02.900238    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s" event={"ID":"752c7a53-2483-43b6-a723-30e32618e40c","Type":"ContainerStarted","Data":"c21c03ca3c6499ff43dfecd9c6cb8f6614f08b557136a71325506f4ee92fc23f"}
Sep 14 03:16:03.809408 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:03.809369    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:16:03.809649 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:03.809578    2574 patch_prober.go:28] interesting pod/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 14 03:16:03.809726 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:03.809650    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s" podUID="752c7a53-2483-43b6-a723-30e32618e40c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 14 03:16:04.133201 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:04.133119    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:16:04.808814 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:04.808781    2574 patch_prober.go:28] interesting pod/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 14 03:16:04.808969 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:04.808851    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s" podUID="752c7a53-2483-43b6-a723-30e32618e40c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 14 03:16:05.809580 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:05.809543    2574 patch_prober.go:28] interesting pod/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 14 03:16:05.810157 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:05.810125    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s" podUID="752c7a53-2483-43b6-a723-30e32618e40c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 14 03:16:06.809156 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:06.809119    2574 patch_prober.go:28] interesting pod/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 14 03:16:06.809367 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:06.809186    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s" podUID="752c7a53-2483-43b6-a723-30e32618e40c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 14 03:16:07.809545 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:07.809479    2574 patch_prober.go:28] interesting pod/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 14 03:16:07.810054 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:07.809570    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s" podUID="752c7a53-2483-43b6-a723-30e32618e40c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 14 03:16:08.939078 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:08.939042    2574 generic.go:356] "Generic (PLEG): container finished" podID="974716b6-453d-4626-8ce9-727784c1e8a5" containerID="55c3e4d713c0457471f84c91391753927c9721b3cd7e46c738317926a5e1ead0" exitCode=0
Sep 14 03:16:08.939646 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:08.939118    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb" event={"ID":"974716b6-453d-4626-8ce9-727784c1e8a5","Type":"ContainerDied","Data":"55c3e4d713c0457471f84c91391753927c9721b3cd7e46c738317926a5e1ead0"}
Sep 14 03:16:08.939730 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:08.939711    2574 scope.go:117] "RemoveContainer" containerID="55c3e4d713c0457471f84c91391753927c9721b3cd7e46c738317926a5e1ead0"
Sep 14 03:16:08.941003 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:08.940972    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:16:08.941124 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:08.941072    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:16:08.941187 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:08.941122    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 03:16:08.996904 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:08.996864    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:16:08.997414 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:08.997360    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:16:08.999126 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:08.999104    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:16:09.947585 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:09.947546    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb" event={"ID":"974716b6-453d-4626-8ce9-727784c1e8a5","Type":"ContainerStarted","Data":"68b11a056503f12aa5ef594eb8e1af3a64b1c8519ce32175f6eef09caca69fb7"}
Sep 14 03:16:10.544560 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:10.544505    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:16:10.545688 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:10.545660    2574 patch_prober.go:28] interesting pod/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 14 03:16:10.545805 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:10.545721    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb" podUID="974716b6-453d-4626-8ce9-727784c1e8a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 14 03:16:11.544731 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:11.544700    2574 patch_prober.go:28] interesting pod/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 14 03:16:11.545098 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:11.544758    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb" podUID="974716b6-453d-4626-8ce9-727784c1e8a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 14 03:16:11.761302 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:11.761265    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:16:12.544736 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:12.544698    2574 patch_prober.go:28] interesting pod/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 14 03:16:12.545180 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:12.544755    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb" podUID="974716b6-453d-4626-8ce9-727784c1e8a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 14 03:16:13.544265 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:13.544233    2574 patch_prober.go:28] interesting pod/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 14 03:16:13.544442 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:13.544286    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb" podUID="974716b6-453d-4626-8ce9-727784c1e8a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 14 03:16:14.544871 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:14.544822    2574 patch_prober.go:28] interesting pod/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 14 03:16:14.545395 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:14.544929    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb" podUID="974716b6-453d-4626-8ce9-727784c1e8a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 14 03:16:15.525539 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:15.525447    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-40c586-8dhhf"]
Sep 14 03:16:15.528066 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:15.528050    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-40c586-8dhhf"
Sep 14 03:16:15.530617 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:15.530597    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:16:15.530747 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:15.530639    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4hf5p\""
Sep 14 03:16:15.536975 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:15.536950    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-40c586-8dhhf"]
Sep 14 03:16:15.671891 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:15.671854    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa174407-2360-4127-bc26-4b768ead53e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-40c586-8dhhf\" (UID: \"aa174407-2360-4127-bc26-4b768ead53e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-40c586-8dhhf"
Sep 14 03:16:15.687652 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:15.687627    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:16:15.689127 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:15.689106    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:16:15.772638 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:15.772604    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa174407-2360-4127-bc26-4b768ead53e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-40c586-8dhhf\" (UID: \"aa174407-2360-4127-bc26-4b768ead53e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-40c586-8dhhf"
Sep 14 03:16:15.773303 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:15.773284    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa174407-2360-4127-bc26-4b768ead53e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-40c586-8dhhf\" (UID: \"aa174407-2360-4127-bc26-4b768ead53e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-40c586-8dhhf"
Sep 14 03:16:15.841190 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:15.841123    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-40c586-8dhhf"
Sep 14 03:16:15.991661 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:15.991634    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz_347cad42-21dd-426e-9694-24a16932e9d7/payload-processing/0.log"
Sep 14 03:16:16.112055 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:16.112032    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-40c586-8dhhf"]
Sep 14 03:16:16.113723 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:16:16.113701    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa174407_2360_4127_bc26_4b768ead53e0.slice/crio-1413167ac899ebe3f39993660254566602d827b7f7dbde0fb29dffcb6a715c13 WatchSource:0}: Error finding container 1413167ac899ebe3f39993660254566602d827b7f7dbde0fb29dffcb6a715c13: Status 404 returned error can't find the container with id 1413167ac899ebe3f39993660254566602d827b7f7dbde0fb29dffcb6a715c13
Sep 14 03:16:16.994170 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:16.994139    2574 generic.go:356] "Generic (PLEG): container finished" podID="aa174407-2360-4127-bc26-4b768ead53e0" containerID="344972224ea75ff484bca2413f2089f8f9e9a564ff9bcdfed8cd5e887111b2de" exitCode=0
Sep 14 03:16:16.994542 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:16.994212    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-40c586-8dhhf" event={"ID":"aa174407-2360-4127-bc26-4b768ead53e0","Type":"ContainerDied","Data":"344972224ea75ff484bca2413f2089f8f9e9a564ff9bcdfed8cd5e887111b2de"}
Sep 14 03:16:16.994542 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:16.994248    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-40c586-8dhhf" event={"ID":"aa174407-2360-4127-bc26-4b768ead53e0","Type":"ContainerStarted","Data":"1413167ac899ebe3f39993660254566602d827b7f7dbde0fb29dffcb6a715c13"}
Sep 14 03:16:18.134771 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:18.134745    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-40c586-8dhhf"
Sep 14 03:16:18.192107 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:18.192086    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa174407-2360-4127-bc26-4b768ead53e0-maas-api-service-ca\") pod \"aa174407-2360-4127-bc26-4b768ead53e0\" (UID: \"aa174407-2360-4127-bc26-4b768ead53e0\") "
Sep 14 03:16:18.192405 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:18.192384    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa174407-2360-4127-bc26-4b768ead53e0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aa174407-2360-4127-bc26-4b768ead53e0" (UID: "aa174407-2360-4127-bc26-4b768ead53e0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:16:18.293766 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:18.293708    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa174407-2360-4127-bc26-4b768ead53e0-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:16:19.007087 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:19.007058    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-40c586-8dhhf"
Sep 14 03:16:19.007250 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:19.007056    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-40c586-8dhhf" event={"ID":"aa174407-2360-4127-bc26-4b768ead53e0","Type":"ContainerDied","Data":"1413167ac899ebe3f39993660254566602d827b7f7dbde0fb29dffcb6a715c13"}
Sep 14 03:16:19.007250 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:19.007179    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1413167ac899ebe3f39993660254566602d827b7f7dbde0fb29dffcb6a715c13"
Sep 14 03:16:27.628484 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.628451    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"]
Sep 14 03:16:27.628897 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.628883    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa174407-2360-4127-bc26-4b768ead53e0" containerName="revoke-keys"
Sep 14 03:16:27.628941 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.628899    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa174407-2360-4127-bc26-4b768ead53e0" containerName="revoke-keys"
Sep 14 03:16:27.628976 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.628971    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa174407-2360-4127-bc26-4b768ead53e0" containerName="revoke-keys"
Sep 14 03:16:27.631994 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.631978    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.637066 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.637046    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-2611af\"/\"openshift-service-ca.crt\""
Sep 14 03:16:27.637200 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.637053    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-2611af\"/\"default-dockercfg-fkq46\""
Sep 14 03:16:27.637200 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.637129    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-2611af\"/\"ipp-route-2611af-kserve-self-signed-certs\""
Sep 14 03:16:27.637200 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.637137    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-2611af\"/\"kube-root-ca.crt\""
Sep 14 03:16:27.649552 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.649529    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"]
Sep 14 03:16:27.781813 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.781787    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-home\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.781942 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.781818    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-tmp-dir\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.781942 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.781838    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71a3fe42-1748-429f-8d6f-536b608d0737-tls-certs\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.781942 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.781866    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9gnn\" (UniqueName: \"kubernetes.io/projected/71a3fe42-1748-429f-8d6f-536b608d0737-kube-api-access-l9gnn\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.781942 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.781914    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-model-cache\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.781942 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.781926    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-dshm\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.883293 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.883209    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-home\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.883293 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.883240    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-tmp-dir\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.883293 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.883261    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71a3fe42-1748-429f-8d6f-536b608d0737-tls-certs\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.883293 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.883284    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9gnn\" (UniqueName: \"kubernetes.io/projected/71a3fe42-1748-429f-8d6f-536b608d0737-kube-api-access-l9gnn\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.883746 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.883325    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-model-cache\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.883746 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.883354    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-dshm\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.883746 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.883638    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-home\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.883746 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.883705    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-tmp-dir\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.883886 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.883772    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-model-cache\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.885806 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.885786    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-dshm\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.886036 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.886018    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71a3fe42-1748-429f-8d6f-536b608d0737-tls-certs\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.891707 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.891684    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9gnn\" (UniqueName: \"kubernetes.io/projected/71a3fe42-1748-429f-8d6f-536b608d0737-kube-api-access-l9gnn\") pod \"ipp-route-2611af-kserve-6c48dc5ddc-tjgv7\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") " pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:27.940952 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:27.940929    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:28.215880 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:28.215853    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"]
Sep 14 03:16:28.217403 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:16:28.217377    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71a3fe42_1748_429f_8d6f_536b608d0737.slice/crio-5672ca3491271d81eaa2e1a80ac6f6c6c35b4aac05c1c48443d4f7c167a7a513 WatchSource:0}: Error finding container 5672ca3491271d81eaa2e1a80ac6f6c6c35b4aac05c1c48443d4f7c167a7a513: Status 404 returned error can't find the container with id 5672ca3491271d81eaa2e1a80ac6f6c6c35b4aac05c1c48443d4f7c167a7a513
Sep 14 03:16:29.062158 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:29.062120    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7" event={"ID":"71a3fe42-1748-429f-8d6f-536b608d0737","Type":"ContainerStarted","Data":"3324c5142ccba53d0d79b467cad8f5dcb28e2220a1b1784ae9e78e1ca85197fd"}
Sep 14 03:16:29.062638 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:29.062166    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7" event={"ID":"71a3fe42-1748-429f-8d6f-536b608d0737","Type":"ContainerStarted","Data":"5672ca3491271d81eaa2e1a80ac6f6c6c35b4aac05c1c48443d4f7c167a7a513"}
Sep 14 03:16:31.419894 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:31.419842    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7" podStartSLOduration=4.419825743 podStartE2EDuration="4.419825743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:16:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:16:29.104099792 +0000 UTC m=+1535.417123610" watchObservedRunningTime="2026-09-14 03:16:31.419825743 +0000 UTC m=+1537.732849599"
Sep 14 03:16:31.421255 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:31.421227    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"]
Sep 14 03:16:31.421528 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:31.421483    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7" podUID="662dade8-9aa2-4025-8ae2-a694448e4f3d" containerName="payload-pre-processing" containerID="cri-o://0f354cc1e10dbc94f311dff4e9763fe916784d1acce306ab7f70d9bc1d35a613" gracePeriod=30
Sep 14 03:16:31.610788 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:31.610719    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-40c586-8dhhf"]
Sep 14 03:16:31.617155 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:31.617124    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-40c586-8dhhf"]
Sep 14 03:16:31.681398 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:31.681348    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"
Sep 14 03:16:31.822743 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:31.822708    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/662dade8-9aa2-4025-8ae2-a694448e4f3d-plugins-config-volume\") pod \"662dade8-9aa2-4025-8ae2-a694448e4f3d\" (UID: \"662dade8-9aa2-4025-8ae2-a694448e4f3d\") "
Sep 14 03:16:31.822912 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:31.822753    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtqwm\" (UniqueName: \"kubernetes.io/projected/662dade8-9aa2-4025-8ae2-a694448e4f3d-kube-api-access-dtqwm\") pod \"662dade8-9aa2-4025-8ae2-a694448e4f3d\" (UID: \"662dade8-9aa2-4025-8ae2-a694448e4f3d\") "
Sep 14 03:16:31.823158 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:31.823136    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/662dade8-9aa2-4025-8ae2-a694448e4f3d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "662dade8-9aa2-4025-8ae2-a694448e4f3d" (UID: "662dade8-9aa2-4025-8ae2-a694448e4f3d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:16:31.824984 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:31.824962    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/662dade8-9aa2-4025-8ae2-a694448e4f3d-kube-api-access-dtqwm" (OuterVolumeSpecName: "kube-api-access-dtqwm") pod "662dade8-9aa2-4025-8ae2-a694448e4f3d" (UID: "662dade8-9aa2-4025-8ae2-a694448e4f3d"). InnerVolumeSpecName "kube-api-access-dtqwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:16:31.924427 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:31.924400    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/662dade8-9aa2-4025-8ae2-a694448e4f3d-plugins-config-volume\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:16:31.924427 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:31.924424    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtqwm\" (UniqueName: \"kubernetes.io/projected/662dade8-9aa2-4025-8ae2-a694448e4f3d-kube-api-access-dtqwm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:16:32.079239 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:32.079206    2574 generic.go:356] "Generic (PLEG): container finished" podID="662dade8-9aa2-4025-8ae2-a694448e4f3d" containerID="0f354cc1e10dbc94f311dff4e9763fe916784d1acce306ab7f70d9bc1d35a613" exitCode=0
Sep 14 03:16:32.079375 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:32.079248    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"
Sep 14 03:16:32.079375 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:32.079298    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7" event={"ID":"662dade8-9aa2-4025-8ae2-a694448e4f3d","Type":"ContainerDied","Data":"0f354cc1e10dbc94f311dff4e9763fe916784d1acce306ab7f70d9bc1d35a613"}
Sep 14 03:16:32.079375 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:32.079349    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7" event={"ID":"662dade8-9aa2-4025-8ae2-a694448e4f3d","Type":"ContainerDied","Data":"babd5e29c506a1fb041536f6398686e7c6ba0b2b9ebb14086fab3f20df2f6f71"}
Sep 14 03:16:32.079375 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:32.079374    2574 scope.go:117] "RemoveContainer" containerID="0f354cc1e10dbc94f311dff4e9763fe916784d1acce306ab7f70d9bc1d35a613"
Sep 14 03:16:32.089196 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:32.089178    2574 scope.go:117] "RemoveContainer" containerID="0f354cc1e10dbc94f311dff4e9763fe916784d1acce306ab7f70d9bc1d35a613"
Sep 14 03:16:32.089506 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:16:32.089483    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f354cc1e10dbc94f311dff4e9763fe916784d1acce306ab7f70d9bc1d35a613\": container with ID starting with 0f354cc1e10dbc94f311dff4e9763fe916784d1acce306ab7f70d9bc1d35a613 not found: ID does not exist" containerID="0f354cc1e10dbc94f311dff4e9763fe916784d1acce306ab7f70d9bc1d35a613"
Sep 14 03:16:32.089647 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:32.089539    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f354cc1e10dbc94f311dff4e9763fe916784d1acce306ab7f70d9bc1d35a613"} err="failed to get container status \"0f354cc1e10dbc94f311dff4e9763fe916784d1acce306ab7f70d9bc1d35a613\": rpc error: code = NotFound desc = could not find container \"0f354cc1e10dbc94f311dff4e9763fe916784d1acce306ab7f70d9bc1d35a613\": container with ID starting with 0f354cc1e10dbc94f311dff4e9763fe916784d1acce306ab7f70d9bc1d35a613 not found: ID does not exist"
Sep 14 03:16:32.109245 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:32.109215    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"]
Sep 14 03:16:32.111317 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:32.111296    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7"]
Sep 14 03:16:32.216304 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:32.216279    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="662dade8-9aa2-4025-8ae2-a694448e4f3d" path="/var/lib/kubelet/pods/662dade8-9aa2-4025-8ae2-a694448e4f3d/volumes"
Sep 14 03:16:32.216676 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:32.216662    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa174407-2360-4127-bc26-4b768ead53e0" path="/var/lib/kubelet/pods/aa174407-2360-4127-bc26-4b768ead53e0/volumes"
Sep 14 03:16:32.620614 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:32.620578    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7" podUID="662dade8-9aa2-4025-8ae2-a694448e4f3d" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.67:9004: i/o timeout"
Sep 14 03:16:33.155205 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:33.155172    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:16:37.941408 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:37.941375    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:37.941892 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:37.941422    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:37.954038 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:37.954018    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:38.131641 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:38.131612    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:16:38.857193 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:38.857154    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"]
Sep 14 03:16:38.857458 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:38.857431    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44" podUID="4f26433d-4127-42a6-b23c-86d89534305b" containerName="istio-proxy" containerID="cri-o://5e7751818234abf696628d1d1a7da4531e10bbdc08fee0ea823e3bc7f2a4372f" gracePeriod=30
Sep 14 03:16:41.499701 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:41.499658    2574 patch_prober.go:28] interesting pod/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 03:16:41.500064 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:41.499718    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44" podUID="4f26433d-4127-42a6-b23c-86d89534305b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 03:16:44.107212 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.107182    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:16:44.146954 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.146872    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-workload-certs\") pod \"4f26433d-4127-42a6-b23c-86d89534305b\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") "
Sep 14 03:16:44.147111 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.146952    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f26433d-4127-42a6-b23c-86d89534305b-istio-podinfo\") pod \"4f26433d-4127-42a6-b23c-86d89534305b\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") "
Sep 14 03:16:44.147111 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.146989    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-workload-socket\") pod \"4f26433d-4127-42a6-b23c-86d89534305b\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") "
Sep 14 03:16:44.147111 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.147048    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-credential-socket\") pod \"4f26433d-4127-42a6-b23c-86d89534305b\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") "
Sep 14 03:16:44.147275 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.147144    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-istio-envoy\") pod \"4f26433d-4127-42a6-b23c-86d89534305b\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") "
Sep 14 03:16:44.147275 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.147166    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4f26433d-4127-42a6-b23c-86d89534305b" (UID: "4f26433d-4127-42a6-b23c-86d89534305b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:16:44.147275 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.147181    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmppn\" (UniqueName: \"kubernetes.io/projected/4f26433d-4127-42a6-b23c-86d89534305b-kube-api-access-zmppn\") pod \"4f26433d-4127-42a6-b23c-86d89534305b\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") "
Sep 14 03:16:44.147275 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.147262    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-istio-data\") pod \"4f26433d-4127-42a6-b23c-86d89534305b\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") "
Sep 14 03:16:44.147487 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.147326    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f26433d-4127-42a6-b23c-86d89534305b-istio-token\") pod \"4f26433d-4127-42a6-b23c-86d89534305b\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") "
Sep 14 03:16:44.147487 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.147365    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f26433d-4127-42a6-b23c-86d89534305b-istiod-ca-cert\") pod \"4f26433d-4127-42a6-b23c-86d89534305b\" (UID: \"4f26433d-4127-42a6-b23c-86d89534305b\") "
Sep 14 03:16:44.147487 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.147395    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4f26433d-4127-42a6-b23c-86d89534305b" (UID: "4f26433d-4127-42a6-b23c-86d89534305b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:16:44.147676 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.147566    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-istio-data" (OuterVolumeSpecName: "istio-data") pod "4f26433d-4127-42a6-b23c-86d89534305b" (UID: "4f26433d-4127-42a6-b23c-86d89534305b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:16:44.147676 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.147578    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4f26433d-4127-42a6-b23c-86d89534305b" (UID: "4f26433d-4127-42a6-b23c-86d89534305b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:16:44.147881 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.147849    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:16:44.147881 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.147874    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f26433d-4127-42a6-b23c-86d89534305b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4f26433d-4127-42a6-b23c-86d89534305b" (UID: "4f26433d-4127-42a6-b23c-86d89534305b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:16:44.148047 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.147888    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:16:44.148047 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.147910    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:16:44.148047 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.147929    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:16:44.150121 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.150087    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4f26433d-4127-42a6-b23c-86d89534305b" (UID: "4f26433d-4127-42a6-b23c-86d89534305b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:16:44.150240 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.150200    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4f26433d-4127-42a6-b23c-86d89534305b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4f26433d-4127-42a6-b23c-86d89534305b" (UID: "4f26433d-4127-42a6-b23c-86d89534305b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:16:44.150309 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.150278    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f26433d-4127-42a6-b23c-86d89534305b-istio-token" (OuterVolumeSpecName: "istio-token") pod "4f26433d-4127-42a6-b23c-86d89534305b" (UID: "4f26433d-4127-42a6-b23c-86d89534305b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:16:44.150309 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.150277    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f26433d-4127-42a6-b23c-86d89534305b-kube-api-access-zmppn" (OuterVolumeSpecName: "kube-api-access-zmppn") pod "4f26433d-4127-42a6-b23c-86d89534305b" (UID: "4f26433d-4127-42a6-b23c-86d89534305b"). InnerVolumeSpecName "kube-api-access-zmppn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:16:44.154501 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.154472    2574 generic.go:356] "Generic (PLEG): container finished" podID="4f26433d-4127-42a6-b23c-86d89534305b" containerID="5e7751818234abf696628d1d1a7da4531e10bbdc08fee0ea823e3bc7f2a4372f" exitCode=0
Sep 14 03:16:44.154628 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.154542    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"
Sep 14 03:16:44.154628 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.154561    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44" event={"ID":"4f26433d-4127-42a6-b23c-86d89534305b","Type":"ContainerDied","Data":"5e7751818234abf696628d1d1a7da4531e10bbdc08fee0ea823e3bc7f2a4372f"}
Sep 14 03:16:44.154628 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.154623    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44" event={"ID":"4f26433d-4127-42a6-b23c-86d89534305b","Type":"ContainerDied","Data":"5b0521510215921550f10d1febe1117f9790ff25e8f569f1dec28e258e8170e5"}
Sep 14 03:16:44.154806 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.154655    2574 scope.go:117] "RemoveContainer" containerID="5e7751818234abf696628d1d1a7da4531e10bbdc08fee0ea823e3bc7f2a4372f"
Sep 14 03:16:44.175963 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.175943    2574 scope.go:117] "RemoveContainer" containerID="4c173396c90e09a0ec4d5a765323353b3d8a041c991cf0ad67b54b5f56136e6d"
Sep 14 03:16:44.184534 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.184497    2574 scope.go:117] "RemoveContainer" containerID="5e7751818234abf696628d1d1a7da4531e10bbdc08fee0ea823e3bc7f2a4372f"
Sep 14 03:16:44.184829 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:16:44.184809    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e7751818234abf696628d1d1a7da4531e10bbdc08fee0ea823e3bc7f2a4372f\": container with ID starting with 5e7751818234abf696628d1d1a7da4531e10bbdc08fee0ea823e3bc7f2a4372f not found: ID does not exist" containerID="5e7751818234abf696628d1d1a7da4531e10bbdc08fee0ea823e3bc7f2a4372f"
Sep 14 03:16:44.184913 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.184851    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e7751818234abf696628d1d1a7da4531e10bbdc08fee0ea823e3bc7f2a4372f"} err="failed to get container status \"5e7751818234abf696628d1d1a7da4531e10bbdc08fee0ea823e3bc7f2a4372f\": rpc error: code = NotFound desc = could not find container \"5e7751818234abf696628d1d1a7da4531e10bbdc08fee0ea823e3bc7f2a4372f\": container with ID starting with 5e7751818234abf696628d1d1a7da4531e10bbdc08fee0ea823e3bc7f2a4372f not found: ID does not exist"
Sep 14 03:16:44.184913 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.184878    2574 scope.go:117] "RemoveContainer" containerID="4c173396c90e09a0ec4d5a765323353b3d8a041c991cf0ad67b54b5f56136e6d"
Sep 14 03:16:44.185172 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:16:44.185143    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c173396c90e09a0ec4d5a765323353b3d8a041c991cf0ad67b54b5f56136e6d\": container with ID starting with 4c173396c90e09a0ec4d5a765323353b3d8a041c991cf0ad67b54b5f56136e6d not found: ID does not exist" containerID="4c173396c90e09a0ec4d5a765323353b3d8a041c991cf0ad67b54b5f56136e6d"
Sep 14 03:16:44.185237 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.185181    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c173396c90e09a0ec4d5a765323353b3d8a041c991cf0ad67b54b5f56136e6d"} err="failed to get container status \"4c173396c90e09a0ec4d5a765323353b3d8a041c991cf0ad67b54b5f56136e6d\": rpc error: code = NotFound desc = could not find container \"4c173396c90e09a0ec4d5a765323353b3d8a041c991cf0ad67b54b5f56136e6d\": container with ID starting with 4c173396c90e09a0ec4d5a765323353b3d8a041c991cf0ad67b54b5f56136e6d not found: ID does not exist"
Sep 14 03:16:44.188739 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.188719    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"]
Sep 14 03:16:44.193392 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.193368    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44"]
Sep 14 03:16:44.216848 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.216824    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f26433d-4127-42a6-b23c-86d89534305b" path="/var/lib/kubelet/pods/4f26433d-4127-42a6-b23c-86d89534305b/volumes"
Sep 14 03:16:44.249325 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.249300    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f26433d-4127-42a6-b23c-86d89534305b-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:16:44.249325 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.249326    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmppn\" (UniqueName: \"kubernetes.io/projected/4f26433d-4127-42a6-b23c-86d89534305b-kube-api-access-zmppn\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:16:44.249578 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.249344    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f26433d-4127-42a6-b23c-86d89534305b-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:16:44.249578 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.249358    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f26433d-4127-42a6-b23c-86d89534305b-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:16:44.249578 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:44.249370    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f26433d-4127-42a6-b23c-86d89534305b-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:16:48.144163 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:16:48.144128    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:17:00.138649 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:00.138613    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:17:02.250950 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:02.250922    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"]
Sep 14 03:17:02.251358 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:02.251123    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb" podUID="974716b6-453d-4626-8ce9-727784c1e8a5" containerName="istio-proxy" containerID="cri-o://68b11a056503f12aa5ef594eb8e1af3a64b1c8519ce32175f6eef09caca69fb7" gracePeriod=30
Sep 14 03:17:06.565533 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:06.565489    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz_347cad42-21dd-426e-9694-24a16932e9d7/payload-processing/0.log"
Sep 14 03:17:06.676785 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:06.676757    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz_347cad42-21dd-426e-9694-24a16932e9d7/payload-processing/0.log"
Sep 14 03:17:07.501685 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.501651    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:17:07.681072 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.680995    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-workload-socket\") pod \"974716b6-453d-4626-8ce9-727784c1e8a5\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") "
Sep 14 03:17:07.681072 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681030    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-istio-envoy\") pod \"974716b6-453d-4626-8ce9-727784c1e8a5\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") "
Sep 14 03:17:07.681620 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681085    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/974716b6-453d-4626-8ce9-727784c1e8a5-istiod-ca-cert\") pod \"974716b6-453d-4626-8ce9-727784c1e8a5\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") "
Sep 14 03:17:07.681620 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681112    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz9l2\" (UniqueName: \"kubernetes.io/projected/974716b6-453d-4626-8ce9-727784c1e8a5-kube-api-access-bz9l2\") pod \"974716b6-453d-4626-8ce9-727784c1e8a5\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") "
Sep 14 03:17:07.681620 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681128    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-credential-socket\") pod \"974716b6-453d-4626-8ce9-727784c1e8a5\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") "
Sep 14 03:17:07.681620 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681175    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-istio-data\") pod \"974716b6-453d-4626-8ce9-727784c1e8a5\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") "
Sep 14 03:17:07.681620 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681199    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/974716b6-453d-4626-8ce9-727784c1e8a5-istio-podinfo\") pod \"974716b6-453d-4626-8ce9-727784c1e8a5\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") "
Sep 14 03:17:07.681620 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681223    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/974716b6-453d-4626-8ce9-727784c1e8a5-istio-token\") pod \"974716b6-453d-4626-8ce9-727784c1e8a5\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") "
Sep 14 03:17:07.681620 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681269    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-workload-certs\") pod \"974716b6-453d-4626-8ce9-727784c1e8a5\" (UID: \"974716b6-453d-4626-8ce9-727784c1e8a5\") "
Sep 14 03:17:07.681620 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681308    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "974716b6-453d-4626-8ce9-727784c1e8a5" (UID: "974716b6-453d-4626-8ce9-727784c1e8a5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:07.681620 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681508    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "974716b6-453d-4626-8ce9-727784c1e8a5" (UID: "974716b6-453d-4626-8ce9-727784c1e8a5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:07.681620 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681558    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-istio-data" (OuterVolumeSpecName: "istio-data") pod "974716b6-453d-4626-8ce9-727784c1e8a5" (UID: "974716b6-453d-4626-8ce9-727784c1e8a5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:07.681620 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681613    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/974716b6-453d-4626-8ce9-727784c1e8a5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "974716b6-453d-4626-8ce9-727784c1e8a5" (UID: "974716b6-453d-4626-8ce9-727784c1e8a5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:07.682179 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681678    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:07.682179 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681765    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "974716b6-453d-4626-8ce9-727784c1e8a5" (UID: "974716b6-453d-4626-8ce9-727784c1e8a5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:07.682179 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681976    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:07.682179 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.681998    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:07.683492 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.683463    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "974716b6-453d-4626-8ce9-727784c1e8a5" (UID: "974716b6-453d-4626-8ce9-727784c1e8a5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:07.683628 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.683608    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/974716b6-453d-4626-8ce9-727784c1e8a5-kube-api-access-bz9l2" (OuterVolumeSpecName: "kube-api-access-bz9l2") pod "974716b6-453d-4626-8ce9-727784c1e8a5" (UID: "974716b6-453d-4626-8ce9-727784c1e8a5"). InnerVolumeSpecName "kube-api-access-bz9l2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:07.684072 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.684046    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/974716b6-453d-4626-8ce9-727784c1e8a5-istio-token" (OuterVolumeSpecName: "istio-token") pod "974716b6-453d-4626-8ce9-727784c1e8a5" (UID: "974716b6-453d-4626-8ce9-727784c1e8a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:07.684177 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.684104    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/974716b6-453d-4626-8ce9-727784c1e8a5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "974716b6-453d-4626-8ce9-727784c1e8a5" (UID: "974716b6-453d-4626-8ce9-727784c1e8a5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:17:07.782443 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.782411    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:07.782443 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.782439    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/974716b6-453d-4626-8ce9-727784c1e8a5-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:07.782656 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.782451    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz9l2\" (UniqueName: \"kubernetes.io/projected/974716b6-453d-4626-8ce9-727784c1e8a5-kube-api-access-bz9l2\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:07.782656 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.782464    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/974716b6-453d-4626-8ce9-727784c1e8a5-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:07.782656 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.782475    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/974716b6-453d-4626-8ce9-727784c1e8a5-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:07.782656 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:07.782486    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/974716b6-453d-4626-8ce9-727784c1e8a5-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:08.294479 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:08.294449    2574 generic.go:356] "Generic (PLEG): container finished" podID="974716b6-453d-4626-8ce9-727784c1e8a5" containerID="68b11a056503f12aa5ef594eb8e1af3a64b1c8519ce32175f6eef09caca69fb7" exitCode=0
Sep 14 03:17:08.294655 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:08.294495    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"
Sep 14 03:17:08.294655 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:08.294543    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb" event={"ID":"974716b6-453d-4626-8ce9-727784c1e8a5","Type":"ContainerDied","Data":"68b11a056503f12aa5ef594eb8e1af3a64b1c8519ce32175f6eef09caca69fb7"}
Sep 14 03:17:08.294655 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:08.294597    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb" event={"ID":"974716b6-453d-4626-8ce9-727784c1e8a5","Type":"ContainerDied","Data":"41abdb0b4e1b3a6b68aafe128e388dc24dbe27f82e116f440e9102b1944a1355"}
Sep 14 03:17:08.294655 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:08.294622    2574 scope.go:117] "RemoveContainer" containerID="68b11a056503f12aa5ef594eb8e1af3a64b1c8519ce32175f6eef09caca69fb7"
Sep 14 03:17:08.303800 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:08.303784    2574 scope.go:117] "RemoveContainer" containerID="55c3e4d713c0457471f84c91391753927c9721b3cd7e46c738317926a5e1ead0"
Sep 14 03:17:08.312036 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:08.312016    2574 scope.go:117] "RemoveContainer" containerID="68b11a056503f12aa5ef594eb8e1af3a64b1c8519ce32175f6eef09caca69fb7"
Sep 14 03:17:08.312311 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:17:08.312293    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68b11a056503f12aa5ef594eb8e1af3a64b1c8519ce32175f6eef09caca69fb7\": container with ID starting with 68b11a056503f12aa5ef594eb8e1af3a64b1c8519ce32175f6eef09caca69fb7 not found: ID does not exist" containerID="68b11a056503f12aa5ef594eb8e1af3a64b1c8519ce32175f6eef09caca69fb7"
Sep 14 03:17:08.312359 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:08.312321    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b11a056503f12aa5ef594eb8e1af3a64b1c8519ce32175f6eef09caca69fb7"} err="failed to get container status \"68b11a056503f12aa5ef594eb8e1af3a64b1c8519ce32175f6eef09caca69fb7\": rpc error: code = NotFound desc = could not find container \"68b11a056503f12aa5ef594eb8e1af3a64b1c8519ce32175f6eef09caca69fb7\": container with ID starting with 68b11a056503f12aa5ef594eb8e1af3a64b1c8519ce32175f6eef09caca69fb7 not found: ID does not exist"
Sep 14 03:17:08.312359 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:08.312337    2574 scope.go:117] "RemoveContainer" containerID="55c3e4d713c0457471f84c91391753927c9721b3cd7e46c738317926a5e1ead0"
Sep 14 03:17:08.312628 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:17:08.312610    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55c3e4d713c0457471f84c91391753927c9721b3cd7e46c738317926a5e1ead0\": container with ID starting with 55c3e4d713c0457471f84c91391753927c9721b3cd7e46c738317926a5e1ead0 not found: ID does not exist" containerID="55c3e4d713c0457471f84c91391753927c9721b3cd7e46c738317926a5e1ead0"
Sep 14 03:17:08.312681 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:08.312638    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55c3e4d713c0457471f84c91391753927c9721b3cd7e46c738317926a5e1ead0"} err="failed to get container status \"55c3e4d713c0457471f84c91391753927c9721b3cd7e46c738317926a5e1ead0\": rpc error: code = NotFound desc = could not find container \"55c3e4d713c0457471f84c91391753927c9721b3cd7e46c738317926a5e1ead0\": container with ID starting with 55c3e4d713c0457471f84c91391753927c9721b3cd7e46c738317926a5e1ead0 not found: ID does not exist"
Sep 14 03:17:08.316959 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:08.316940    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"]
Sep 14 03:17:08.323751 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:08.323731    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb"]
Sep 14 03:17:10.216575 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:10.216543    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="974716b6-453d-4626-8ce9-727784c1e8a5" path="/var/lib/kubelet/pods/974716b6-453d-4626-8ce9-727784c1e8a5/volumes"
Sep 14 03:17:16.254730 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.254704    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"]
Sep 14 03:17:16.255143 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.255126    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f26433d-4127-42a6-b23c-86d89534305b" containerName="istio-proxy"
Sep 14 03:17:16.255201 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.255146    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f26433d-4127-42a6-b23c-86d89534305b" containerName="istio-proxy"
Sep 14 03:17:16.255201 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.255155    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="974716b6-453d-4626-8ce9-727784c1e8a5" containerName="istio-proxy"
Sep 14 03:17:16.255201 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.255160    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="974716b6-453d-4626-8ce9-727784c1e8a5" containerName="istio-proxy"
Sep 14 03:17:16.255201 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.255176    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="662dade8-9aa2-4025-8ae2-a694448e4f3d" containerName="payload-pre-processing"
Sep 14 03:17:16.255201 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.255182    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="662dade8-9aa2-4025-8ae2-a694448e4f3d" containerName="payload-pre-processing"
Sep 14 03:17:16.255201 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.255188    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f26433d-4127-42a6-b23c-86d89534305b" containerName="istio-proxy"
Sep 14 03:17:16.255201 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.255193    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f26433d-4127-42a6-b23c-86d89534305b" containerName="istio-proxy"
Sep 14 03:17:16.255447 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.255248    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="974716b6-453d-4626-8ce9-727784c1e8a5" containerName="istio-proxy"
Sep 14 03:17:16.255447 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.255256    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="662dade8-9aa2-4025-8ae2-a694448e4f3d" containerName="payload-pre-processing"
Sep 14 03:17:16.255447 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.255262    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f26433d-4127-42a6-b23c-86d89534305b" containerName="istio-proxy"
Sep 14 03:17:16.255447 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.255268    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f26433d-4127-42a6-b23c-86d89534305b" containerName="istio-proxy"
Sep 14 03:17:16.259680 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.259661    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"
Sep 14 03:17:16.262175 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.262159    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-a35f4b\""
Sep 14 03:17:16.262248 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.262220    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-a35f4b-dockercfg-svgjk\""
Sep 14 03:17:16.267267 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.267243    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"]
Sep 14 03:17:16.367540 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.367051    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f232699f-a5d5-4fde-91a6-ee7723fde74e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz\" (UID: \"f232699f-a5d5-4fde-91a6-ee7723fde74e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"
Sep 14 03:17:16.367540 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.367240    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxm5v\" (UniqueName: \"kubernetes.io/projected/f232699f-a5d5-4fde-91a6-ee7723fde74e-kube-api-access-rxm5v\") pod \"payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz\" (UID: \"f232699f-a5d5-4fde-91a6-ee7723fde74e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"
Sep 14 03:17:16.465013 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.464980    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-6e87d0-29822595-sdb6m"]
Sep 14 03:17:16.468052 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.468024    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxm5v\" (UniqueName: \"kubernetes.io/projected/f232699f-a5d5-4fde-91a6-ee7723fde74e-kube-api-access-rxm5v\") pod \"payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz\" (UID: \"f232699f-a5d5-4fde-91a6-ee7723fde74e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"
Sep 14 03:17:16.468207 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.468074    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f232699f-a5d5-4fde-91a6-ee7723fde74e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz\" (UID: \"f232699f-a5d5-4fde-91a6-ee7723fde74e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"
Sep 14 03:17:16.468353 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.468331    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-6e87d0-29822595-sdb6m"]
Sep 14 03:17:16.468817 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.468799    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f232699f-a5d5-4fde-91a6-ee7723fde74e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz\" (UID: \"f232699f-a5d5-4fde-91a6-ee7723fde74e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"
Sep 14 03:17:16.480956 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.480923    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxm5v\" (UniqueName: \"kubernetes.io/projected/f232699f-a5d5-4fde-91a6-ee7723fde74e-kube-api-access-rxm5v\") pod \"payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz\" (UID: \"f232699f-a5d5-4fde-91a6-ee7723fde74e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"
Sep 14 03:17:16.570643 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.570565    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"
Sep 14 03:17:16.865665 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:17:16.865611    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf232699f_a5d5_4fde_91a6_ee7723fde74e.slice/crio-a1f25147ed2fe3f22e6c860177ca0dc194fad09908a73fe93bc2447feceadafd WatchSource:0}: Error finding container a1f25147ed2fe3f22e6c860177ca0dc194fad09908a73fe93bc2447feceadafd: Status 404 returned error can't find the container with id a1f25147ed2fe3f22e6c860177ca0dc194fad09908a73fe93bc2447feceadafd
Sep 14 03:17:16.867351 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:16.867322    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"]
Sep 14 03:17:17.355638 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:17.355415    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz" event={"ID":"f232699f-a5d5-4fde-91a6-ee7723fde74e","Type":"ContainerStarted","Data":"495d44f650823075635b88e78eee4ccad5ef0d6f3b49aab57d436ba26de9693b"}
Sep 14 03:17:17.355638 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:17.355555    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz" event={"ID":"f232699f-a5d5-4fde-91a6-ee7723fde74e","Type":"ContainerStarted","Data":"a1f25147ed2fe3f22e6c860177ca0dc194fad09908a73fe93bc2447feceadafd"}
Sep 14 03:17:17.355638 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:17.355585    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"
Sep 14 03:17:17.376186 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:17.376138    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz" podStartSLOduration=1.376122209 podStartE2EDuration="1.376122209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:17:17.374414933 +0000 UTC m=+1583.687438752" watchObservedRunningTime="2026-09-14 03:17:17.376122209 +0000 UTC m=+1583.689146028"
Sep 14 03:17:18.216155 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:18.216121    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14ec788a-5fc5-4553-b94c-54a4ce6ae306" path="/var/lib/kubelet/pods/14ec788a-5fc5-4553-b94c-54a4ce6ae306/volumes"
Sep 14 03:17:20.743252 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:20.743210    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:17:23.814171 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:23.814138    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"]
Sep 14 03:17:23.814569 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:23.814389    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq" podUID="d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" containerName="istio-proxy" containerID="cri-o://a6d02bed96cb55ac7b7e9c922f61a38762895a48286ce28712f860a3fd52c8b7" gracePeriod=30
Sep 14 03:17:28.363797 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:28.363750    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"
Sep 14 03:17:29.079649 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.079592    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:17:29.193607 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.193583    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhxh8\" (UniqueName: \"kubernetes.io/projected/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-kube-api-access-dhxh8\") pod \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") "
Sep 14 03:17:29.193762 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.193622    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istiod-ca-cert\") pod \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") "
Sep 14 03:17:29.193762 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.193660    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-data\") pod \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") "
Sep 14 03:17:29.193762 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.193716    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-token\") pod \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") "
Sep 14 03:17:29.193762 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.193757    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-workload-certs\") pod \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") "
Sep 14 03:17:29.193946 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.193802    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-workload-socket\") pod \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") "
Sep 14 03:17:29.193946 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.193867    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-envoy\") pod \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") "
Sep 14 03:17:29.193946 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.193891    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-podinfo\") pod \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") "
Sep 14 03:17:29.194103 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.193948    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-credential-socket\") pod \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\" (UID: \"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e\") "
Sep 14 03:17:29.194103 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.194001    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-data" (OuterVolumeSpecName: "istio-data") pod "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" (UID: "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:29.194103 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.194042    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" (UID: "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:29.194103 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.194051    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" (UID: "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:29.194301 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.194145    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" (UID: "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:29.194301 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.194287    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" (UID: "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:29.194416 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.194345    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:29.194416 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.194363    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:29.194416 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.194372    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:29.194416 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.194380    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:29.196031 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.196005    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-kube-api-access-dhxh8" (OuterVolumeSpecName: "kube-api-access-dhxh8") pod "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" (UID: "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e"). InnerVolumeSpecName "kube-api-access-dhxh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:29.196220 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.196193    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-token" (OuterVolumeSpecName: "istio-token") pod "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" (UID: "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:29.196551 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.196503    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" (UID: "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:29.196551 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.196534    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" (UID: "d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:17:29.295265 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.295243    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:29.295265 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.295270    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:29.295405 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.295283    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:29.295405 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.295297    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhxh8\" (UniqueName: \"kubernetes.io/projected/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-kube-api-access-dhxh8\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:29.295405 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.295310    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:29.422376 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.422314    2574 generic.go:356] "Generic (PLEG): container finished" podID="d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" containerID="a6d02bed96cb55ac7b7e9c922f61a38762895a48286ce28712f860a3fd52c8b7" exitCode=0
Sep 14 03:17:29.422376 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.422365    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"
Sep 14 03:17:29.422784 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.422397    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq" event={"ID":"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e","Type":"ContainerDied","Data":"a6d02bed96cb55ac7b7e9c922f61a38762895a48286ce28712f860a3fd52c8b7"}
Sep 14 03:17:29.422784 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.422445    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq" event={"ID":"d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e","Type":"ContainerDied","Data":"2b66a19aa8319bc2066b31381ec3c9177652f45dec1f4599ebd861ae2460fcbd"}
Sep 14 03:17:29.422784 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.422467    2574 scope.go:117] "RemoveContainer" containerID="a6d02bed96cb55ac7b7e9c922f61a38762895a48286ce28712f860a3fd52c8b7"
Sep 14 03:17:29.432565 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.432547    2574 scope.go:117] "RemoveContainer" containerID="6f1aa3c1ec0b84b6b705a497a60654b59346407c815d60114d91c90d8452cec0"
Sep 14 03:17:29.441620 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.441600    2574 scope.go:117] "RemoveContainer" containerID="a6d02bed96cb55ac7b7e9c922f61a38762895a48286ce28712f860a3fd52c8b7"
Sep 14 03:17:29.441867 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:17:29.441848    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6d02bed96cb55ac7b7e9c922f61a38762895a48286ce28712f860a3fd52c8b7\": container with ID starting with a6d02bed96cb55ac7b7e9c922f61a38762895a48286ce28712f860a3fd52c8b7 not found: ID does not exist" containerID="a6d02bed96cb55ac7b7e9c922f61a38762895a48286ce28712f860a3fd52c8b7"
Sep 14 03:17:29.441926 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.441876    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6d02bed96cb55ac7b7e9c922f61a38762895a48286ce28712f860a3fd52c8b7"} err="failed to get container status \"a6d02bed96cb55ac7b7e9c922f61a38762895a48286ce28712f860a3fd52c8b7\": rpc error: code = NotFound desc = could not find container \"a6d02bed96cb55ac7b7e9c922f61a38762895a48286ce28712f860a3fd52c8b7\": container with ID starting with a6d02bed96cb55ac7b7e9c922f61a38762895a48286ce28712f860a3fd52c8b7 not found: ID does not exist"
Sep 14 03:17:29.441926 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.441894    2574 scope.go:117] "RemoveContainer" containerID="6f1aa3c1ec0b84b6b705a497a60654b59346407c815d60114d91c90d8452cec0"
Sep 14 03:17:29.442147 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:17:29.442126    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f1aa3c1ec0b84b6b705a497a60654b59346407c815d60114d91c90d8452cec0\": container with ID starting with 6f1aa3c1ec0b84b6b705a497a60654b59346407c815d60114d91c90d8452cec0 not found: ID does not exist" containerID="6f1aa3c1ec0b84b6b705a497a60654b59346407c815d60114d91c90d8452cec0"
Sep 14 03:17:29.442211 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.442153    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f1aa3c1ec0b84b6b705a497a60654b59346407c815d60114d91c90d8452cec0"} err="failed to get container status \"6f1aa3c1ec0b84b6b705a497a60654b59346407c815d60114d91c90d8452cec0\": rpc error: code = NotFound desc = could not find container \"6f1aa3c1ec0b84b6b705a497a60654b59346407c815d60114d91c90d8452cec0\": container with ID starting with 6f1aa3c1ec0b84b6b705a497a60654b59346407c815d60114d91c90d8452cec0 not found: ID does not exist"
Sep 14 03:17:29.445164 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.445144    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"]
Sep 14 03:17:29.449438 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:29.449418    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq"]
Sep 14 03:17:30.216227 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:30.216198    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" path="/var/lib/kubelet/pods/d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e/volumes"
Sep 14 03:17:41.336617 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:41.336581    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:17:42.429973 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:42.429941    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"]
Sep 14 03:17:42.430435 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:42.430162    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz" podUID="f232699f-a5d5-4fde-91a6-ee7723fde74e" containerName="payload-processing" containerID="cri-o://495d44f650823075635b88e78eee4ccad5ef0d6f3b49aab57d436ba26de9693b" gracePeriod=30
Sep 14 03:17:42.692005 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:42.691946    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"
Sep 14 03:17:42.821687 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:42.821658    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxm5v\" (UniqueName: \"kubernetes.io/projected/f232699f-a5d5-4fde-91a6-ee7723fde74e-kube-api-access-rxm5v\") pod \"f232699f-a5d5-4fde-91a6-ee7723fde74e\" (UID: \"f232699f-a5d5-4fde-91a6-ee7723fde74e\") "
Sep 14 03:17:42.821890 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:42.821778    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f232699f-a5d5-4fde-91a6-ee7723fde74e-plugins-config-volume\") pod \"f232699f-a5d5-4fde-91a6-ee7723fde74e\" (UID: \"f232699f-a5d5-4fde-91a6-ee7723fde74e\") "
Sep 14 03:17:42.822168 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:42.822142    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f232699f-a5d5-4fde-91a6-ee7723fde74e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f232699f-a5d5-4fde-91a6-ee7723fde74e" (UID: "f232699f-a5d5-4fde-91a6-ee7723fde74e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:42.824059 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:42.824037    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f232699f-a5d5-4fde-91a6-ee7723fde74e-kube-api-access-rxm5v" (OuterVolumeSpecName: "kube-api-access-rxm5v") pod "f232699f-a5d5-4fde-91a6-ee7723fde74e" (UID: "f232699f-a5d5-4fde-91a6-ee7723fde74e"). InnerVolumeSpecName "kube-api-access-rxm5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:42.922451 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:42.922424    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f232699f-a5d5-4fde-91a6-ee7723fde74e-plugins-config-volume\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:42.922451 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:42.922448    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxm5v\" (UniqueName: \"kubernetes.io/projected/f232699f-a5d5-4fde-91a6-ee7723fde74e-kube-api-access-rxm5v\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:43.505801 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:43.505763    2574 generic.go:356] "Generic (PLEG): container finished" podID="f232699f-a5d5-4fde-91a6-ee7723fde74e" containerID="495d44f650823075635b88e78eee4ccad5ef0d6f3b49aab57d436ba26de9693b" exitCode=0
Sep 14 03:17:43.506289 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:43.505814    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"
Sep 14 03:17:43.506289 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:43.505875    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz" event={"ID":"f232699f-a5d5-4fde-91a6-ee7723fde74e","Type":"ContainerDied","Data":"495d44f650823075635b88e78eee4ccad5ef0d6f3b49aab57d436ba26de9693b"}
Sep 14 03:17:43.506289 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:43.505943    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz" event={"ID":"f232699f-a5d5-4fde-91a6-ee7723fde74e","Type":"ContainerDied","Data":"a1f25147ed2fe3f22e6c860177ca0dc194fad09908a73fe93bc2447feceadafd"}
Sep 14 03:17:43.506289 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:43.505975    2574 scope.go:117] "RemoveContainer" containerID="495d44f650823075635b88e78eee4ccad5ef0d6f3b49aab57d436ba26de9693b"
Sep 14 03:17:43.516331 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:43.516313    2574 scope.go:117] "RemoveContainer" containerID="495d44f650823075635b88e78eee4ccad5ef0d6f3b49aab57d436ba26de9693b"
Sep 14 03:17:43.516596 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:17:43.516575    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"495d44f650823075635b88e78eee4ccad5ef0d6f3b49aab57d436ba26de9693b\": container with ID starting with 495d44f650823075635b88e78eee4ccad5ef0d6f3b49aab57d436ba26de9693b not found: ID does not exist" containerID="495d44f650823075635b88e78eee4ccad5ef0d6f3b49aab57d436ba26de9693b"
Sep 14 03:17:43.516668 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:43.516607    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"495d44f650823075635b88e78eee4ccad5ef0d6f3b49aab57d436ba26de9693b"} err="failed to get container status \"495d44f650823075635b88e78eee4ccad5ef0d6f3b49aab57d436ba26de9693b\": rpc error: code = NotFound desc = could not find container \"495d44f650823075635b88e78eee4ccad5ef0d6f3b49aab57d436ba26de9693b\": container with ID starting with 495d44f650823075635b88e78eee4ccad5ef0d6f3b49aab57d436ba26de9693b not found: ID does not exist"
Sep 14 03:17:43.531052 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:43.531027    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"]
Sep 14 03:17:43.536619 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:43.536600    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz"]
Sep 14 03:17:44.216396 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:44.216360    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f232699f-a5d5-4fde-91a6-ee7723fde74e" path="/var/lib/kubelet/pods/f232699f-a5d5-4fde-91a6-ee7723fde74e/volumes"
Sep 14 03:17:45.021926 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:45.021892    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"]
Sep 14 03:17:45.022346 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:45.022164    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl" podUID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" containerName="istio-proxy" containerID="cri-o://442d65f1e5039d54651c9462e411b8a908718efab86532deac41ac172958143c" gracePeriod=30
Sep 14 03:17:49.864984 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:49.864940    2574 patch_prober.go:28] interesting pod/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 03:17:49.865569 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:49.865503    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl" podUID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 03:17:50.269209 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.268078    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:17:50.387466 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.387397    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-workload-certs\") pod \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") "
Sep 14 03:17:50.387466 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.387435    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-data\") pod \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") "
Sep 14 03:17:50.387466 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.387462    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgvj5\" (UniqueName: \"kubernetes.io/projected/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-kube-api-access-jgvj5\") pod \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") "
Sep 14 03:17:50.387712 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.387556    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-credential-socket\") pod \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") "
Sep 14 03:17:50.387712 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.387588    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-podinfo\") pod \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") "
Sep 14 03:17:50.387712 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.387639    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-envoy\") pod \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") "
Sep 14 03:17:50.387712 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.387669    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istiod-ca-cert\") pod \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") "
Sep 14 03:17:50.387916 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.387713    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-workload-socket\") pod \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") "
Sep 14 03:17:50.387916 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.387725    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bf9a5b9b-5dc8-4811-9091-5b8f8508244c" (UID: "bf9a5b9b-5dc8-4811-9091-5b8f8508244c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:50.387916 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.387734    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-data" (OuterVolumeSpecName: "istio-data") pod "bf9a5b9b-5dc8-4811-9091-5b8f8508244c" (UID: "bf9a5b9b-5dc8-4811-9091-5b8f8508244c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:50.387916 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.387779    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-token\") pod \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\" (UID: \"bf9a5b9b-5dc8-4811-9091-5b8f8508244c\") "
Sep 14 03:17:50.388115 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.387986    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bf9a5b9b-5dc8-4811-9091-5b8f8508244c" (UID: "bf9a5b9b-5dc8-4811-9091-5b8f8508244c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:50.388115 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.388011    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bf9a5b9b-5dc8-4811-9091-5b8f8508244c" (UID: "bf9a5b9b-5dc8-4811-9091-5b8f8508244c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:50.388115 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.388081    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bf9a5b9b-5dc8-4811-9091-5b8f8508244c" (UID: "bf9a5b9b-5dc8-4811-9091-5b8f8508244c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:50.388276 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.388122    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:50.388276 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.388142    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:50.388276 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.388157    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:50.388276 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.388170    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:50.390086 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.390043    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bf9a5b9b-5dc8-4811-9091-5b8f8508244c" (UID: "bf9a5b9b-5dc8-4811-9091-5b8f8508244c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:50.390193 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.390127    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-token" (OuterVolumeSpecName: "istio-token") pod "bf9a5b9b-5dc8-4811-9091-5b8f8508244c" (UID: "bf9a5b9b-5dc8-4811-9091-5b8f8508244c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:50.390382 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.390354    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bf9a5b9b-5dc8-4811-9091-5b8f8508244c" (UID: "bf9a5b9b-5dc8-4811-9091-5b8f8508244c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:17:50.390476 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.390360    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-kube-api-access-jgvj5" (OuterVolumeSpecName: "kube-api-access-jgvj5") pod "bf9a5b9b-5dc8-4811-9091-5b8f8508244c" (UID: "bf9a5b9b-5dc8-4811-9091-5b8f8508244c"). InnerVolumeSpecName "kube-api-access-jgvj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:50.489627 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.489593    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:50.489627 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.489620    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:50.489627 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.489632    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:50.489840 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.489644    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:50.489840 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.489654    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgvj5\" (UniqueName: \"kubernetes.io/projected/bf9a5b9b-5dc8-4811-9091-5b8f8508244c-kube-api-access-jgvj5\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:50.544652 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.544622    2574 generic.go:356] "Generic (PLEG): container finished" podID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" containerID="442d65f1e5039d54651c9462e411b8a908718efab86532deac41ac172958143c" exitCode=0
Sep 14 03:17:50.544805 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.544665    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"
Sep 14 03:17:50.544805 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.544695    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl" event={"ID":"bf9a5b9b-5dc8-4811-9091-5b8f8508244c","Type":"ContainerDied","Data":"442d65f1e5039d54651c9462e411b8a908718efab86532deac41ac172958143c"}
Sep 14 03:17:50.544805 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.544760    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl" event={"ID":"bf9a5b9b-5dc8-4811-9091-5b8f8508244c","Type":"ContainerDied","Data":"b44b4dde7ee11c66a0e097a92c80e4b103223a6aba30a371e1b98336ad8a9bdd"}
Sep 14 03:17:50.544805 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.544783    2574 scope.go:117] "RemoveContainer" containerID="442d65f1e5039d54651c9462e411b8a908718efab86532deac41ac172958143c"
Sep 14 03:17:50.554302 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.554286    2574 scope.go:117] "RemoveContainer" containerID="450acff468342625292199e64178442371059fba5d118a4634c0188bc9d2cbe3"
Sep 14 03:17:50.562285 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.562268    2574 scope.go:117] "RemoveContainer" containerID="442d65f1e5039d54651c9462e411b8a908718efab86532deac41ac172958143c"
Sep 14 03:17:50.562557 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:17:50.562538    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"442d65f1e5039d54651c9462e411b8a908718efab86532deac41ac172958143c\": container with ID starting with 442d65f1e5039d54651c9462e411b8a908718efab86532deac41ac172958143c not found: ID does not exist" containerID="442d65f1e5039d54651c9462e411b8a908718efab86532deac41ac172958143c"
Sep 14 03:17:50.562644 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.562570    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"442d65f1e5039d54651c9462e411b8a908718efab86532deac41ac172958143c"} err="failed to get container status \"442d65f1e5039d54651c9462e411b8a908718efab86532deac41ac172958143c\": rpc error: code = NotFound desc = could not find container \"442d65f1e5039d54651c9462e411b8a908718efab86532deac41ac172958143c\": container with ID starting with 442d65f1e5039d54651c9462e411b8a908718efab86532deac41ac172958143c not found: ID does not exist"
Sep 14 03:17:50.562644 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.562596    2574 scope.go:117] "RemoveContainer" containerID="450acff468342625292199e64178442371059fba5d118a4634c0188bc9d2cbe3"
Sep 14 03:17:50.562844 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:17:50.562825    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"450acff468342625292199e64178442371059fba5d118a4634c0188bc9d2cbe3\": container with ID starting with 450acff468342625292199e64178442371059fba5d118a4634c0188bc9d2cbe3 not found: ID does not exist" containerID="450acff468342625292199e64178442371059fba5d118a4634c0188bc9d2cbe3"
Sep 14 03:17:50.562886 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.562850    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"450acff468342625292199e64178442371059fba5d118a4634c0188bc9d2cbe3"} err="failed to get container status \"450acff468342625292199e64178442371059fba5d118a4634c0188bc9d2cbe3\": rpc error: code = NotFound desc = could not find container \"450acff468342625292199e64178442371059fba5d118a4634c0188bc9d2cbe3\": container with ID starting with 450acff468342625292199e64178442371059fba5d118a4634c0188bc9d2cbe3 not found: ID does not exist"
Sep 14 03:17:50.573699 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.573679    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"]
Sep 14 03:17:50.580308 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:50.580285    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl"]
Sep 14 03:17:52.215651 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:52.215623    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" path="/var/lib/kubelet/pods/bf9a5b9b-5dc8-4811-9091-5b8f8508244c/volumes"
Sep 14 03:17:53.327740 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.327711    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4"]
Sep 14 03:17:53.328107 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328072    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="974716b6-453d-4626-8ce9-727784c1e8a5" containerName="istio-proxy"
Sep 14 03:17:53.328107 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328082    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="974716b6-453d-4626-8ce9-727784c1e8a5" containerName="istio-proxy"
Sep 14 03:17:53.328107 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328099    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f232699f-a5d5-4fde-91a6-ee7723fde74e" containerName="payload-processing"
Sep 14 03:17:53.328107 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328104    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f232699f-a5d5-4fde-91a6-ee7723fde74e" containerName="payload-processing"
Sep 14 03:17:53.328243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328116    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" containerName="istio-proxy"
Sep 14 03:17:53.328243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328122    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" containerName="istio-proxy"
Sep 14 03:17:53.328243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328137    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" containerName="istio-proxy"
Sep 14 03:17:53.328243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328142    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" containerName="istio-proxy"
Sep 14 03:17:53.328243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328148    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" containerName="istio-proxy"
Sep 14 03:17:53.328243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328153    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" containerName="istio-proxy"
Sep 14 03:17:53.328243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328161    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" containerName="istio-proxy"
Sep 14 03:17:53.328243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328166    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" containerName="istio-proxy"
Sep 14 03:17:53.328243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328220    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" containerName="istio-proxy"
Sep 14 03:17:53.328243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328227    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1b7ad87-3c3a-4dfd-b3f0-6a9e10a67d9e" containerName="istio-proxy"
Sep 14 03:17:53.328243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328235    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f232699f-a5d5-4fde-91a6-ee7723fde74e" containerName="payload-processing"
Sep 14 03:17:53.328243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328243    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" containerName="istio-proxy"
Sep 14 03:17:53.328629 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328249    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf9a5b9b-5dc8-4811-9091-5b8f8508244c" containerName="istio-proxy"
Sep 14 03:17:53.328629 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.328254    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="974716b6-453d-4626-8ce9-727784c1e8a5" containerName="istio-proxy"
Sep 14 03:17:53.332533 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.332496    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4"
Sep 14 03:17:53.336233 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.336213    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:17:53.336702 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.336687    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4hf5p\""
Sep 14 03:17:53.346196 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.346172    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4"]
Sep 14 03:17:53.410137 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.410114    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82241387-97c7-4cbf-9a52-4ac93c8a04b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4\" (UID: \"82241387-97c7-4cbf-9a52-4ac93c8a04b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4"
Sep 14 03:17:53.511275 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.511243    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82241387-97c7-4cbf-9a52-4ac93c8a04b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4\" (UID: \"82241387-97c7-4cbf-9a52-4ac93c8a04b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4"
Sep 14 03:17:53.511915 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.511891    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82241387-97c7-4cbf-9a52-4ac93c8a04b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4\" (UID: \"82241387-97c7-4cbf-9a52-4ac93c8a04b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4"
Sep 14 03:17:53.641935 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.641872    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4"
Sep 14 03:17:53.919628 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:53.919606    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4"]
Sep 14 03:17:53.921803 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:17:53.921774    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82241387_97c7_4cbf_9a52_4ac93c8a04b3.slice/crio-6715f58d9994bf6322a9d361b6faff00bcf55d0d7f8e53dc6fbe86bc624697db WatchSource:0}: Error finding container 6715f58d9994bf6322a9d361b6faff00bcf55d0d7f8e53dc6fbe86bc624697db: Status 404 returned error can't find the container with id 6715f58d9994bf6322a9d361b6faff00bcf55d0d7f8e53dc6fbe86bc624697db
Sep 14 03:17:54.449559 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:54.449531    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:17:54.569849 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:54.569820    2574 generic.go:356] "Generic (PLEG): container finished" podID="82241387-97c7-4cbf-9a52-4ac93c8a04b3" containerID="8fbef5bdbf2e095dc10326339ecede3d74d924c5b35b64052f8a4588186aeef1" exitCode=0
Sep 14 03:17:54.569969 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:54.569902    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4" event={"ID":"82241387-97c7-4cbf-9a52-4ac93c8a04b3","Type":"ContainerDied","Data":"8fbef5bdbf2e095dc10326339ecede3d74d924c5b35b64052f8a4588186aeef1"}
Sep 14 03:17:54.569969 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:54.569960    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4" event={"ID":"82241387-97c7-4cbf-9a52-4ac93c8a04b3","Type":"ContainerStarted","Data":"6715f58d9994bf6322a9d361b6faff00bcf55d0d7f8e53dc6fbe86bc624697db"}
Sep 14 03:17:55.709017 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:55.708993    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4"
Sep 14 03:17:55.734300 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:55.734276    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82241387-97c7-4cbf-9a52-4ac93c8a04b3-maas-api-service-ca\") pod \"82241387-97c7-4cbf-9a52-4ac93c8a04b3\" (UID: \"82241387-97c7-4cbf-9a52-4ac93c8a04b3\") "
Sep 14 03:17:55.734585 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:55.734565    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82241387-97c7-4cbf-9a52-4ac93c8a04b3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "82241387-97c7-4cbf-9a52-4ac93c8a04b3" (UID: "82241387-97c7-4cbf-9a52-4ac93c8a04b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:55.835107 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:55.835046    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82241387-97c7-4cbf-9a52-4ac93c8a04b3-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:17:56.582168 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:56.582143    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4"
Sep 14 03:17:56.582344 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:56.582141    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4" event={"ID":"82241387-97c7-4cbf-9a52-4ac93c8a04b3","Type":"ContainerDied","Data":"6715f58d9994bf6322a9d361b6faff00bcf55d0d7f8e53dc6fbe86bc624697db"}
Sep 14 03:17:56.582344 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:17:56.582279    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6715f58d9994bf6322a9d361b6faff00bcf55d0d7f8e53dc6fbe86bc624697db"
Sep 14 03:18:06.543051 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:06.543015    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:18:09.198583 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:09.198548    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4"]
Sep 14 03:18:09.201941 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:09.201915    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-2611af-69mm4"]
Sep 14 03:18:10.216303 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:10.216270    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82241387-97c7-4cbf-9a52-4ac93c8a04b3" path="/var/lib/kubelet/pods/82241387-97c7-4cbf-9a52-4ac93c8a04b3/volumes"
Sep 14 03:18:14.542989 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:14.542953    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"]
Sep 14 03:18:19.242857 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:19.242827    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:18:29.562656 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.562605    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7" podUID="71a3fe42-1748-429f-8d6f-536b608d0737" containerName="main" containerID="cri-o://3324c5142ccba53d0d79b467cad8f5dcb28e2220a1b1784ae9e78e1ca85197fd" gracePeriod=45
Sep 14 03:18:29.760725 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.760698    2574 generic.go:356] "Generic (PLEG): container finished" podID="71a3fe42-1748-429f-8d6f-536b608d0737" containerID="3324c5142ccba53d0d79b467cad8f5dcb28e2220a1b1784ae9e78e1ca85197fd" exitCode=0
Sep 14 03:18:29.760885 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.760781    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7" event={"ID":"71a3fe42-1748-429f-8d6f-536b608d0737","Type":"ContainerDied","Data":"3324c5142ccba53d0d79b467cad8f5dcb28e2220a1b1784ae9e78e1ca85197fd"}
Sep 14 03:18:29.817247 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.817192    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:18:29.924829 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.924801    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-dshm\") pod \"71a3fe42-1748-429f-8d6f-536b608d0737\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") "
Sep 14 03:18:29.925003 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.924889    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9gnn\" (UniqueName: \"kubernetes.io/projected/71a3fe42-1748-429f-8d6f-536b608d0737-kube-api-access-l9gnn\") pod \"71a3fe42-1748-429f-8d6f-536b608d0737\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") "
Sep 14 03:18:29.925003 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.924934    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71a3fe42-1748-429f-8d6f-536b608d0737-tls-certs\") pod \"71a3fe42-1748-429f-8d6f-536b608d0737\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") "
Sep 14 03:18:29.925003 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.924974    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-model-cache\") pod \"71a3fe42-1748-429f-8d6f-536b608d0737\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") "
Sep 14 03:18:29.925003 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.925001    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-home\") pod \"71a3fe42-1748-429f-8d6f-536b608d0737\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") "
Sep 14 03:18:29.925241 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.925060    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-tmp-dir\") pod \"71a3fe42-1748-429f-8d6f-536b608d0737\" (UID: \"71a3fe42-1748-429f-8d6f-536b608d0737\") "
Sep 14 03:18:29.925320 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.925276    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-home" (OuterVolumeSpecName: "home") pod "71a3fe42-1748-429f-8d6f-536b608d0737" (UID: "71a3fe42-1748-429f-8d6f-536b608d0737"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:18:29.925320 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.925307    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-model-cache" (OuterVolumeSpecName: "model-cache") pod "71a3fe42-1748-429f-8d6f-536b608d0737" (UID: "71a3fe42-1748-429f-8d6f-536b608d0737"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:18:29.925422 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.925340    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "71a3fe42-1748-429f-8d6f-536b608d0737" (UID: "71a3fe42-1748-429f-8d6f-536b608d0737"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:18:29.925422 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.925399    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:18:29.925422 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.925417    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:18:29.927216 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.927189    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-dshm" (OuterVolumeSpecName: "dshm") pod "71a3fe42-1748-429f-8d6f-536b608d0737" (UID: "71a3fe42-1748-429f-8d6f-536b608d0737"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:18:29.927329 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.927261    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71a3fe42-1748-429f-8d6f-536b608d0737-kube-api-access-l9gnn" (OuterVolumeSpecName: "kube-api-access-l9gnn") pod "71a3fe42-1748-429f-8d6f-536b608d0737" (UID: "71a3fe42-1748-429f-8d6f-536b608d0737"). InnerVolumeSpecName "kube-api-access-l9gnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:18:29.927329 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:29.927270    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71a3fe42-1748-429f-8d6f-536b608d0737-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "71a3fe42-1748-429f-8d6f-536b608d0737" (UID: "71a3fe42-1748-429f-8d6f-536b608d0737"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:18:30.026181 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:30.026149    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:18:30.026181 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:30.026177    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9gnn\" (UniqueName: \"kubernetes.io/projected/71a3fe42-1748-429f-8d6f-536b608d0737-kube-api-access-l9gnn\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:18:30.026181 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:30.026188    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71a3fe42-1748-429f-8d6f-536b608d0737-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:18:30.026405 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:30.026200    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71a3fe42-1748-429f-8d6f-536b608d0737-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:18:30.767060 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:30.767030    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7" event={"ID":"71a3fe42-1748-429f-8d6f-536b608d0737","Type":"ContainerDied","Data":"5672ca3491271d81eaa2e1a80ac6f6c6c35b4aac05c1c48443d4f7c167a7a513"}
Sep 14 03:18:30.767060 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:30.767050    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"
Sep 14 03:18:30.767060 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:30.767074    2574 scope.go:117] "RemoveContainer" containerID="3324c5142ccba53d0d79b467cad8f5dcb28e2220a1b1784ae9e78e1ca85197fd"
Sep 14 03:18:30.783311 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:30.783291    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"]
Sep 14 03:18:30.787215 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:30.787193    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-2611af/ipp-route-2611af-kserve-6c48dc5ddc-tjgv7"]
Sep 14 03:18:32.215629 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:18:32.215594    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71a3fe42-1748-429f-8d6f-536b608d0737" path="/var/lib/kubelet/pods/71a3fe42-1748-429f-8d6f-536b608d0737/volumes"
Sep 14 03:19:04.410413 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:04.410382    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"]
Sep 14 03:19:04.410967 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:04.410610    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z" podUID="a5159351-6b1d-4ddd-a385-e7ea69d07591" containerName="istio-proxy" containerID="cri-o://06fa898d26da54eaa2af0d28828722b825d0cb80e650665ed1eb17c3d08860c8" gracePeriod=30
Sep 14 03:19:09.663294 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.663269    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:19:09.759799 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.759778    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-podinfo\") pod \"a5159351-6b1d-4ddd-a385-e7ea69d07591\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") "
Sep 14 03:19:09.759918 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.759814    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8lrv\" (UniqueName: \"kubernetes.io/projected/a5159351-6b1d-4ddd-a385-e7ea69d07591-kube-api-access-d8lrv\") pod \"a5159351-6b1d-4ddd-a385-e7ea69d07591\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") "
Sep 14 03:19:09.759918 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.759857    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-workload-certs\") pod \"a5159351-6b1d-4ddd-a385-e7ea69d07591\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") "
Sep 14 03:19:09.759918 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.759894    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-workload-socket\") pod \"a5159351-6b1d-4ddd-a385-e7ea69d07591\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") "
Sep 14 03:19:09.760030 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.759955    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-envoy\") pod \"a5159351-6b1d-4ddd-a385-e7ea69d07591\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") "
Sep 14 03:19:09.760030 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.759984    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-credential-socket\") pod \"a5159351-6b1d-4ddd-a385-e7ea69d07591\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") "
Sep 14 03:19:09.760131 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.760044    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5159351-6b1d-4ddd-a385-e7ea69d07591-istiod-ca-cert\") pod \"a5159351-6b1d-4ddd-a385-e7ea69d07591\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") "
Sep 14 03:19:09.760131 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.760074    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-data\") pod \"a5159351-6b1d-4ddd-a385-e7ea69d07591\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") "
Sep 14 03:19:09.760236 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.760204    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a5159351-6b1d-4ddd-a385-e7ea69d07591" (UID: "a5159351-6b1d-4ddd-a385-e7ea69d07591"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:19:09.760236 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.760213    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a5159351-6b1d-4ddd-a385-e7ea69d07591" (UID: "a5159351-6b1d-4ddd-a385-e7ea69d07591"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:19:09.760340 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.760270    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-token\") pod \"a5159351-6b1d-4ddd-a385-e7ea69d07591\" (UID: \"a5159351-6b1d-4ddd-a385-e7ea69d07591\") "
Sep 14 03:19:09.760706 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.760361    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a5159351-6b1d-4ddd-a385-e7ea69d07591" (UID: "a5159351-6b1d-4ddd-a385-e7ea69d07591"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:19:09.760827 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.760553    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5159351-6b1d-4ddd-a385-e7ea69d07591-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a5159351-6b1d-4ddd-a385-e7ea69d07591" (UID: "a5159351-6b1d-4ddd-a385-e7ea69d07591"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:19:09.760827 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.760500    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-data" (OuterVolumeSpecName: "istio-data") pod "a5159351-6b1d-4ddd-a385-e7ea69d07591" (UID: "a5159351-6b1d-4ddd-a385-e7ea69d07591"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:19:09.761025 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.760956    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:09.761025 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.760977    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:09.761025 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.760992    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:09.761025 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.761007    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5159351-6b1d-4ddd-a385-e7ea69d07591-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:09.761025 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.761021    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:09.762166 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.762135    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5159351-6b1d-4ddd-a385-e7ea69d07591-kube-api-access-d8lrv" (OuterVolumeSpecName: "kube-api-access-d8lrv") pod "a5159351-6b1d-4ddd-a385-e7ea69d07591" (UID: "a5159351-6b1d-4ddd-a385-e7ea69d07591"). InnerVolumeSpecName "kube-api-access-d8lrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:19:09.762274 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.762250    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a5159351-6b1d-4ddd-a385-e7ea69d07591" (UID: "a5159351-6b1d-4ddd-a385-e7ea69d07591"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:19:09.762339 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.762316    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a5159351-6b1d-4ddd-a385-e7ea69d07591" (UID: "a5159351-6b1d-4ddd-a385-e7ea69d07591"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:19:09.762448 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.762433    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-token" (OuterVolumeSpecName: "istio-token") pod "a5159351-6b1d-4ddd-a385-e7ea69d07591" (UID: "a5159351-6b1d-4ddd-a385-e7ea69d07591"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:19:09.861830 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.861808    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:09.861830 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.861828    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:09.861967 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.861838    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8lrv\" (UniqueName: \"kubernetes.io/projected/a5159351-6b1d-4ddd-a385-e7ea69d07591-kube-api-access-d8lrv\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:09.861967 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.861847    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5159351-6b1d-4ddd-a385-e7ea69d07591-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:09.984019 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.983990    2574 generic.go:356] "Generic (PLEG): container finished" podID="a5159351-6b1d-4ddd-a385-e7ea69d07591" containerID="06fa898d26da54eaa2af0d28828722b825d0cb80e650665ed1eb17c3d08860c8" exitCode=0
Sep 14 03:19:09.984150 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.984039    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"
Sep 14 03:19:09.984150 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.984079    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z" event={"ID":"a5159351-6b1d-4ddd-a385-e7ea69d07591","Type":"ContainerDied","Data":"06fa898d26da54eaa2af0d28828722b825d0cb80e650665ed1eb17c3d08860c8"}
Sep 14 03:19:09.984150 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.984130    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z" event={"ID":"a5159351-6b1d-4ddd-a385-e7ea69d07591","Type":"ContainerDied","Data":"be20ccd065ed33efaf572b8d0e67fd349d4d08a7fab0f39ca218e2d7b3beb9a1"}
Sep 14 03:19:09.984264 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.984152    2574 scope.go:117] "RemoveContainer" containerID="06fa898d26da54eaa2af0d28828722b825d0cb80e650665ed1eb17c3d08860c8"
Sep 14 03:19:09.993699 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:09.993686    2574 scope.go:117] "RemoveContainer" containerID="92b132094e3289191e00761fe19caf53a48426635013d37f306b610946c025ae"
Sep 14 03:19:10.001887 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:10.001872    2574 scope.go:117] "RemoveContainer" containerID="06fa898d26da54eaa2af0d28828722b825d0cb80e650665ed1eb17c3d08860c8"
Sep 14 03:19:10.002163 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:19:10.002146    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06fa898d26da54eaa2af0d28828722b825d0cb80e650665ed1eb17c3d08860c8\": container with ID starting with 06fa898d26da54eaa2af0d28828722b825d0cb80e650665ed1eb17c3d08860c8 not found: ID does not exist" containerID="06fa898d26da54eaa2af0d28828722b825d0cb80e650665ed1eb17c3d08860c8"
Sep 14 03:19:10.002211 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:10.002169    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06fa898d26da54eaa2af0d28828722b825d0cb80e650665ed1eb17c3d08860c8"} err="failed to get container status \"06fa898d26da54eaa2af0d28828722b825d0cb80e650665ed1eb17c3d08860c8\": rpc error: code = NotFound desc = could not find container \"06fa898d26da54eaa2af0d28828722b825d0cb80e650665ed1eb17c3d08860c8\": container with ID starting with 06fa898d26da54eaa2af0d28828722b825d0cb80e650665ed1eb17c3d08860c8 not found: ID does not exist"
Sep 14 03:19:10.002211 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:10.002185    2574 scope.go:117] "RemoveContainer" containerID="92b132094e3289191e00761fe19caf53a48426635013d37f306b610946c025ae"
Sep 14 03:19:10.002427 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:19:10.002412    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92b132094e3289191e00761fe19caf53a48426635013d37f306b610946c025ae\": container with ID starting with 92b132094e3289191e00761fe19caf53a48426635013d37f306b610946c025ae not found: ID does not exist" containerID="92b132094e3289191e00761fe19caf53a48426635013d37f306b610946c025ae"
Sep 14 03:19:10.002482 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:10.002430    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92b132094e3289191e00761fe19caf53a48426635013d37f306b610946c025ae"} err="failed to get container status \"92b132094e3289191e00761fe19caf53a48426635013d37f306b610946c025ae\": rpc error: code = NotFound desc = could not find container \"92b132094e3289191e00761fe19caf53a48426635013d37f306b610946c025ae\": container with ID starting with 92b132094e3289191e00761fe19caf53a48426635013d37f306b610946c025ae not found: ID does not exist"
Sep 14 03:19:10.006467 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:10.006437    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"]
Sep 14 03:19:10.011950 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:10.011906    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z"]
Sep 14 03:19:10.217022 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:10.216960    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5159351-6b1d-4ddd-a385-e7ea69d07591" path="/var/lib/kubelet/pods/a5159351-6b1d-4ddd-a385-e7ea69d07591/volumes"
Sep 14 03:19:15.317388 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.317359    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"]
Sep 14 03:19:15.317851 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.317573    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz" podUID="347cad42-21dd-426e-9694-24a16932e9d7" containerName="payload-processing" containerID="cri-o://764b31c0b769a9337343a71ec82756e4e76ea5bfce04552e5b5bb3818ddaeba7" gracePeriod=30
Sep 14 03:19:15.325773 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.325745    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"]
Sep 14 03:19:15.325986 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.325964    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4" podUID="fcf16ad4-b620-4fa6-a95f-9a899f6e58dd" containerName="payload-pre-processing" containerID="cri-o://35e56e3c870bdc4464c07d2b21341a25740cac075a9b2d97f83296eb1cbecbfd" gracePeriod=30
Sep 14 03:19:15.578270 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.578213    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"
Sep 14 03:19:15.604764 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.604739    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"
Sep 14 03:19:15.712583 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.712551    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csb2s\" (UniqueName: \"kubernetes.io/projected/fcf16ad4-b620-4fa6-a95f-9a899f6e58dd-kube-api-access-csb2s\") pod \"fcf16ad4-b620-4fa6-a95f-9a899f6e58dd\" (UID: \"fcf16ad4-b620-4fa6-a95f-9a899f6e58dd\") "
Sep 14 03:19:15.712780 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.712598    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf16ad4-b620-4fa6-a95f-9a899f6e58dd-plugins-config-volume\") pod \"fcf16ad4-b620-4fa6-a95f-9a899f6e58dd\" (UID: \"fcf16ad4-b620-4fa6-a95f-9a899f6e58dd\") "
Sep 14 03:19:15.712780 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.712645    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gnmn\" (UniqueName: \"kubernetes.io/projected/347cad42-21dd-426e-9694-24a16932e9d7-kube-api-access-2gnmn\") pod \"347cad42-21dd-426e-9694-24a16932e9d7\" (UID: \"347cad42-21dd-426e-9694-24a16932e9d7\") "
Sep 14 03:19:15.712780 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.712726    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/347cad42-21dd-426e-9694-24a16932e9d7-plugins-config-volume\") pod \"347cad42-21dd-426e-9694-24a16932e9d7\" (UID: \"347cad42-21dd-426e-9694-24a16932e9d7\") "
Sep 14 03:19:15.713045 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.713016    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcf16ad4-b620-4fa6-a95f-9a899f6e58dd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fcf16ad4-b620-4fa6-a95f-9a899f6e58dd" (UID: "fcf16ad4-b620-4fa6-a95f-9a899f6e58dd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:19:15.713157 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.713122    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/347cad42-21dd-426e-9694-24a16932e9d7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "347cad42-21dd-426e-9694-24a16932e9d7" (UID: "347cad42-21dd-426e-9694-24a16932e9d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:19:15.714917 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.714897    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/347cad42-21dd-426e-9694-24a16932e9d7-kube-api-access-2gnmn" (OuterVolumeSpecName: "kube-api-access-2gnmn") pod "347cad42-21dd-426e-9694-24a16932e9d7" (UID: "347cad42-21dd-426e-9694-24a16932e9d7"). InnerVolumeSpecName "kube-api-access-2gnmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:19:15.715017 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.714921    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcf16ad4-b620-4fa6-a95f-9a899f6e58dd-kube-api-access-csb2s" (OuterVolumeSpecName: "kube-api-access-csb2s") pod "fcf16ad4-b620-4fa6-a95f-9a899f6e58dd" (UID: "fcf16ad4-b620-4fa6-a95f-9a899f6e58dd"). InnerVolumeSpecName "kube-api-access-csb2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:19:15.813658 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.813634    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/347cad42-21dd-426e-9694-24a16932e9d7-plugins-config-volume\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:15.813658 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.813654    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csb2s\" (UniqueName: \"kubernetes.io/projected/fcf16ad4-b620-4fa6-a95f-9a899f6e58dd-kube-api-access-csb2s\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:15.813779 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.813663    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf16ad4-b620-4fa6-a95f-9a899f6e58dd-plugins-config-volume\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:15.813779 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:15.813672    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gnmn\" (UniqueName: \"kubernetes.io/projected/347cad42-21dd-426e-9694-24a16932e9d7-kube-api-access-2gnmn\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:16.019018 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.018983    2574 generic.go:356] "Generic (PLEG): container finished" podID="fcf16ad4-b620-4fa6-a95f-9a899f6e58dd" containerID="35e56e3c870bdc4464c07d2b21341a25740cac075a9b2d97f83296eb1cbecbfd" exitCode=0
Sep 14 03:19:16.019204 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.019031    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"
Sep 14 03:19:16.019204 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.019073    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4" event={"ID":"fcf16ad4-b620-4fa6-a95f-9a899f6e58dd","Type":"ContainerDied","Data":"35e56e3c870bdc4464c07d2b21341a25740cac075a9b2d97f83296eb1cbecbfd"}
Sep 14 03:19:16.019204 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.019144    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4" event={"ID":"fcf16ad4-b620-4fa6-a95f-9a899f6e58dd","Type":"ContainerDied","Data":"98baba3e110d99e812e2785259dd1697714ed4c48734c12caadfe96b2001b5c7"}
Sep 14 03:19:16.019204 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.019175    2574 scope.go:117] "RemoveContainer" containerID="35e56e3c870bdc4464c07d2b21341a25740cac075a9b2d97f83296eb1cbecbfd"
Sep 14 03:19:16.020538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.020489    2574 generic.go:356] "Generic (PLEG): container finished" podID="347cad42-21dd-426e-9694-24a16932e9d7" containerID="764b31c0b769a9337343a71ec82756e4e76ea5bfce04552e5b5bb3818ddaeba7" exitCode=0
Sep 14 03:19:16.020646 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.020554    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz" event={"ID":"347cad42-21dd-426e-9694-24a16932e9d7","Type":"ContainerDied","Data":"764b31c0b769a9337343a71ec82756e4e76ea5bfce04552e5b5bb3818ddaeba7"}
Sep 14 03:19:16.020646 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.020599    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz" event={"ID":"347cad42-21dd-426e-9694-24a16932e9d7","Type":"ContainerDied","Data":"0aab68c6f8933c77d7c173010cca5f922f427a37b43a0699233610a910da71a1"}
Sep 14 03:19:16.020646 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.020561    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"
Sep 14 03:19:16.028913 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.028896    2574 scope.go:117] "RemoveContainer" containerID="35e56e3c870bdc4464c07d2b21341a25740cac075a9b2d97f83296eb1cbecbfd"
Sep 14 03:19:16.029176 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:19:16.029157    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35e56e3c870bdc4464c07d2b21341a25740cac075a9b2d97f83296eb1cbecbfd\": container with ID starting with 35e56e3c870bdc4464c07d2b21341a25740cac075a9b2d97f83296eb1cbecbfd not found: ID does not exist" containerID="35e56e3c870bdc4464c07d2b21341a25740cac075a9b2d97f83296eb1cbecbfd"
Sep 14 03:19:16.029242 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.029181    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35e56e3c870bdc4464c07d2b21341a25740cac075a9b2d97f83296eb1cbecbfd"} err="failed to get container status \"35e56e3c870bdc4464c07d2b21341a25740cac075a9b2d97f83296eb1cbecbfd\": rpc error: code = NotFound desc = could not find container \"35e56e3c870bdc4464c07d2b21341a25740cac075a9b2d97f83296eb1cbecbfd\": container with ID starting with 35e56e3c870bdc4464c07d2b21341a25740cac075a9b2d97f83296eb1cbecbfd not found: ID does not exist"
Sep 14 03:19:16.029242 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.029196    2574 scope.go:117] "RemoveContainer" containerID="764b31c0b769a9337343a71ec82756e4e76ea5bfce04552e5b5bb3818ddaeba7"
Sep 14 03:19:16.037302 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.037288    2574 scope.go:117] "RemoveContainer" containerID="764b31c0b769a9337343a71ec82756e4e76ea5bfce04552e5b5bb3818ddaeba7"
Sep 14 03:19:16.037598 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:19:16.037570    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"764b31c0b769a9337343a71ec82756e4e76ea5bfce04552e5b5bb3818ddaeba7\": container with ID starting with 764b31c0b769a9337343a71ec82756e4e76ea5bfce04552e5b5bb3818ddaeba7 not found: ID does not exist" containerID="764b31c0b769a9337343a71ec82756e4e76ea5bfce04552e5b5bb3818ddaeba7"
Sep 14 03:19:16.037670 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.037608    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"764b31c0b769a9337343a71ec82756e4e76ea5bfce04552e5b5bb3818ddaeba7"} err="failed to get container status \"764b31c0b769a9337343a71ec82756e4e76ea5bfce04552e5b5bb3818ddaeba7\": rpc error: code = NotFound desc = could not find container \"764b31c0b769a9337343a71ec82756e4e76ea5bfce04552e5b5bb3818ddaeba7\": container with ID starting with 764b31c0b769a9337343a71ec82756e4e76ea5bfce04552e5b5bb3818ddaeba7 not found: ID does not exist"
Sep 14 03:19:16.043314 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.043293    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"]
Sep 14 03:19:16.047496 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.047476    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz"]
Sep 14 03:19:16.057559 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.057505    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"]
Sep 14 03:19:16.063968 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.063945    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4"]
Sep 14 03:19:16.215423 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.215399    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="347cad42-21dd-426e-9694-24a16932e9d7" path="/var/lib/kubelet/pods/347cad42-21dd-426e-9694-24a16932e9d7/volumes"
Sep 14 03:19:16.215771 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:16.215757    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcf16ad4-b620-4fa6-a95f-9a899f6e58dd" path="/var/lib/kubelet/pods/fcf16ad4-b620-4fa6-a95f-9a899f6e58dd/volumes"
Sep 14 03:19:18.728939 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:18.728908    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:19:22.557616 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.557585    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"]
Sep 14 03:19:22.558015 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.557774    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s" podUID="752c7a53-2483-43b6-a723-30e32618e40c" containerName="istio-proxy" containerID="cri-o://c21c03ca3c6499ff43dfecd9c6cb8f6614f08b557136a71325506f4ee92fc23f" gracePeriod=30
Sep 14 03:19:22.881140 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881067    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7"]
Sep 14 03:19:22.881396 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881384    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5159351-6b1d-4ddd-a385-e7ea69d07591" containerName="istio-proxy"
Sep 14 03:19:22.881449 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881397    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5159351-6b1d-4ddd-a385-e7ea69d07591" containerName="istio-proxy"
Sep 14 03:19:22.881449 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881405    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="347cad42-21dd-426e-9694-24a16932e9d7" containerName="payload-processing"
Sep 14 03:19:22.881449 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881411    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="347cad42-21dd-426e-9694-24a16932e9d7" containerName="payload-processing"
Sep 14 03:19:22.881449 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881420    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5159351-6b1d-4ddd-a385-e7ea69d07591" containerName="istio-proxy"
Sep 14 03:19:22.881449 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881425    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5159351-6b1d-4ddd-a385-e7ea69d07591" containerName="istio-proxy"
Sep 14 03:19:22.881449 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881434    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71a3fe42-1748-429f-8d6f-536b608d0737" containerName="main"
Sep 14 03:19:22.881449 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881439    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="71a3fe42-1748-429f-8d6f-536b608d0737" containerName="main"
Sep 14 03:19:22.881449 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881450    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82241387-97c7-4cbf-9a52-4ac93c8a04b3" containerName="revoke-keys"
Sep 14 03:19:22.881764 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881455    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="82241387-97c7-4cbf-9a52-4ac93c8a04b3" containerName="revoke-keys"
Sep 14 03:19:22.881764 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881466    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcf16ad4-b620-4fa6-a95f-9a899f6e58dd" containerName="payload-pre-processing"
Sep 14 03:19:22.881764 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881471    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcf16ad4-b620-4fa6-a95f-9a899f6e58dd" containerName="payload-pre-processing"
Sep 14 03:19:22.881764 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881549    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5159351-6b1d-4ddd-a385-e7ea69d07591" containerName="istio-proxy"
Sep 14 03:19:22.881764 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881558    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5159351-6b1d-4ddd-a385-e7ea69d07591" containerName="istio-proxy"
Sep 14 03:19:22.881764 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881566    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="347cad42-21dd-426e-9694-24a16932e9d7" containerName="payload-processing"
Sep 14 03:19:22.881764 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881572    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="82241387-97c7-4cbf-9a52-4ac93c8a04b3" containerName="revoke-keys"
Sep 14 03:19:22.881764 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881578    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="71a3fe42-1748-429f-8d6f-536b608d0737" containerName="main"
Sep 14 03:19:22.881764 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.881586    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcf16ad4-b620-4fa6-a95f-9a899f6e58dd" containerName="payload-pre-processing"
Sep 14 03:19:22.888486 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.888462    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7"
Sep 14 03:19:22.891915 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.891892    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4hf5p\""
Sep 14 03:19:22.892040 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.891949    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:19:22.896614 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.896596    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7"]
Sep 14 03:19:22.967358 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:22.967335    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d75a77-cc0f-4a2f-8838-176c08b59f95-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7\" (UID: \"b6d75a77-cc0f-4a2f-8838-176c08b59f95\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7"
Sep 14 03:19:23.068029 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:23.068003    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d75a77-cc0f-4a2f-8838-176c08b59f95-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7\" (UID: \"b6d75a77-cc0f-4a2f-8838-176c08b59f95\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7"
Sep 14 03:19:23.068671 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:23.068652    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d75a77-cc0f-4a2f-8838-176c08b59f95-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7\" (UID: \"b6d75a77-cc0f-4a2f-8838-176c08b59f95\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7"
Sep 14 03:19:23.199958 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:23.199882    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7"
Sep 14 03:19:23.481747 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:23.481717    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7"]
Sep 14 03:19:23.482423 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:19:23.482399    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d75a77_cc0f_4a2f_8838_176c08b59f95.slice/crio-e2f7a94d4513f9c4459e6137f150b1d2985296a9423483269a0432b6da05a344 WatchSource:0}: Error finding container e2f7a94d4513f9c4459e6137f150b1d2985296a9423483269a0432b6da05a344: Status 404 returned error can't find the container with id e2f7a94d4513f9c4459e6137f150b1d2985296a9423483269a0432b6da05a344
Sep 14 03:19:23.999830 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:23.999794    2574 patch_prober.go:28] interesting pod/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 03:19:24.000196 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:23.999852    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s" podUID="752c7a53-2483-43b6-a723-30e32618e40c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 03:19:24.065588 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:24.065556    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7" event={"ID":"b6d75a77-cc0f-4a2f-8838-176c08b59f95","Type":"ContainerStarted","Data":"34337d18db9393109b95af5499b3e18f6e332d345247bc46a702e3ebbc36bc8f"}
Sep 14 03:19:24.065732 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:24.065618    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7" event={"ID":"b6d75a77-cc0f-4a2f-8838-176c08b59f95","Type":"ContainerStarted","Data":"e2f7a94d4513f9c4459e6137f150b1d2985296a9423483269a0432b6da05a344"}
Sep 14 03:19:24.083300 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:24.083257    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7" podStartSLOduration=2.083238624 podStartE2EDuration="2.083238624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:19:24.083038422 +0000 UTC m=+1710.396062241" watchObservedRunningTime="2026-09-14 03:19:24.083238624 +0000 UTC m=+1710.396262447"
Sep 14 03:19:25.071868 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:25.071836    2574 generic.go:356] "Generic (PLEG): container finished" podID="b6d75a77-cc0f-4a2f-8838-176c08b59f95" containerID="34337d18db9393109b95af5499b3e18f6e332d345247bc46a702e3ebbc36bc8f" exitCode=0
Sep 14 03:19:25.072260 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:25.071930    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7" event={"ID":"b6d75a77-cc0f-4a2f-8838-176c08b59f95","Type":"ContainerDied","Data":"34337d18db9393109b95af5499b3e18f6e332d345247bc46a702e3ebbc36bc8f"}
Sep 14 03:19:26.210330 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:26.210307    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7"
Sep 14 03:19:26.296572 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:26.296544    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d75a77-cc0f-4a2f-8838-176c08b59f95-maas-api-service-ca\") pod \"b6d75a77-cc0f-4a2f-8838-176c08b59f95\" (UID: \"b6d75a77-cc0f-4a2f-8838-176c08b59f95\") "
Sep 14 03:19:26.296896 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:26.296876    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6d75a77-cc0f-4a2f-8838-176c08b59f95-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b6d75a77-cc0f-4a2f-8838-176c08b59f95" (UID: "b6d75a77-cc0f-4a2f-8838-176c08b59f95"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:19:26.397919 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:26.397854    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d75a77-cc0f-4a2f-8838-176c08b59f95-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:27.082724 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.082693    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7"
Sep 14 03:19:27.082724 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.082708    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7" event={"ID":"b6d75a77-cc0f-4a2f-8838-176c08b59f95","Type":"ContainerDied","Data":"e2f7a94d4513f9c4459e6137f150b1d2985296a9423483269a0432b6da05a344"}
Sep 14 03:19:27.082930 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.082755    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2f7a94d4513f9c4459e6137f150b1d2985296a9423483269a0432b6da05a344"
Sep 14 03:19:27.803083 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.803059    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:19:27.911286 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.911221    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/752c7a53-2483-43b6-a723-30e32618e40c-istio-token\") pod \"752c7a53-2483-43b6-a723-30e32618e40c\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") "
Sep 14 03:19:27.911286 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.911253    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/752c7a53-2483-43b6-a723-30e32618e40c-istio-podinfo\") pod \"752c7a53-2483-43b6-a723-30e32618e40c\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") "
Sep 14 03:19:27.911286 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.911276    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9h9kp\" (UniqueName: \"kubernetes.io/projected/752c7a53-2483-43b6-a723-30e32618e40c-kube-api-access-9h9kp\") pod \"752c7a53-2483-43b6-a723-30e32618e40c\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") "
Sep 14 03:19:27.911564 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.911295    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-credential-socket\") pod \"752c7a53-2483-43b6-a723-30e32618e40c\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") "
Sep 14 03:19:27.911564 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.911389    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-istio-data\") pod \"752c7a53-2483-43b6-a723-30e32618e40c\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") "
Sep 14 03:19:27.911564 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.911444    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-workload-socket\") pod \"752c7a53-2483-43b6-a723-30e32618e40c\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") "
Sep 14 03:19:27.911564 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.911500    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-workload-certs\") pod \"752c7a53-2483-43b6-a723-30e32618e40c\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") "
Sep 14 03:19:27.911564 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.911550    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-istio-envoy\") pod \"752c7a53-2483-43b6-a723-30e32618e40c\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") "
Sep 14 03:19:27.911842 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.911575    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "752c7a53-2483-43b6-a723-30e32618e40c" (UID: "752c7a53-2483-43b6-a723-30e32618e40c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:19:27.911842 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.911644    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/752c7a53-2483-43b6-a723-30e32618e40c-istiod-ca-cert\") pod \"752c7a53-2483-43b6-a723-30e32618e40c\" (UID: \"752c7a53-2483-43b6-a723-30e32618e40c\") "
Sep 14 03:19:27.911842 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.911653    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-istio-data" (OuterVolumeSpecName: "istio-data") pod "752c7a53-2483-43b6-a723-30e32618e40c" (UID: "752c7a53-2483-43b6-a723-30e32618e40c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:19:27.911842 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.911751    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "752c7a53-2483-43b6-a723-30e32618e40c" (UID: "752c7a53-2483-43b6-a723-30e32618e40c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:19:27.911842 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.911776    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "752c7a53-2483-43b6-a723-30e32618e40c" (UID: "752c7a53-2483-43b6-a723-30e32618e40c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:19:27.912267 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.911979    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:27.912267 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.911996    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:27.912267 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.912022    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:27.912267 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.912036    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:27.912267 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.912234    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/752c7a53-2483-43b6-a723-30e32618e40c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "752c7a53-2483-43b6-a723-30e32618e40c" (UID: "752c7a53-2483-43b6-a723-30e32618e40c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:19:27.913687 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.913659    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/752c7a53-2483-43b6-a723-30e32618e40c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "752c7a53-2483-43b6-a723-30e32618e40c" (UID: "752c7a53-2483-43b6-a723-30e32618e40c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:19:27.913773 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.913716    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/752c7a53-2483-43b6-a723-30e32618e40c-istio-token" (OuterVolumeSpecName: "istio-token") pod "752c7a53-2483-43b6-a723-30e32618e40c" (UID: "752c7a53-2483-43b6-a723-30e32618e40c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:19:27.913773 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.913716    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/752c7a53-2483-43b6-a723-30e32618e40c-kube-api-access-9h9kp" (OuterVolumeSpecName: "kube-api-access-9h9kp") pod "752c7a53-2483-43b6-a723-30e32618e40c" (UID: "752c7a53-2483-43b6-a723-30e32618e40c"). InnerVolumeSpecName "kube-api-access-9h9kp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:19:27.913871 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:27.913853    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "752c7a53-2483-43b6-a723-30e32618e40c" (UID: "752c7a53-2483-43b6-a723-30e32618e40c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:19:28.013344 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.013316    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/752c7a53-2483-43b6-a723-30e32618e40c-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:28.013344 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.013339    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/752c7a53-2483-43b6-a723-30e32618e40c-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:28.013344 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.013349    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h9kp\" (UniqueName: \"kubernetes.io/projected/752c7a53-2483-43b6-a723-30e32618e40c-kube-api-access-9h9kp\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:28.013588 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.013357    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/752c7a53-2483-43b6-a723-30e32618e40c-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:28.013588 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.013365    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/752c7a53-2483-43b6-a723-30e32618e40c-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:28.095786 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.095753    2574 generic.go:356] "Generic (PLEG): container finished" podID="752c7a53-2483-43b6-a723-30e32618e40c" containerID="c21c03ca3c6499ff43dfecd9c6cb8f6614f08b557136a71325506f4ee92fc23f" exitCode=0
Sep 14 03:19:28.095918 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.095819    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"
Sep 14 03:19:28.095918 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.095830    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s" event={"ID":"752c7a53-2483-43b6-a723-30e32618e40c","Type":"ContainerDied","Data":"c21c03ca3c6499ff43dfecd9c6cb8f6614f08b557136a71325506f4ee92fc23f"}
Sep 14 03:19:28.095918 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.095910    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s" event={"ID":"752c7a53-2483-43b6-a723-30e32618e40c","Type":"ContainerDied","Data":"2e4d4818ba075d28112cceea20da327f7b2901df45c3c4252d524ab8fcb6f4ba"}
Sep 14 03:19:28.096024 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.095933    2574 scope.go:117] "RemoveContainer" containerID="c21c03ca3c6499ff43dfecd9c6cb8f6614f08b557136a71325506f4ee92fc23f"
Sep 14 03:19:28.105582 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.105567    2574 scope.go:117] "RemoveContainer" containerID="cceff37ea00daba6c65dfb1e192feb049211425b5fa8bc97713bef38a0b00514"
Sep 14 03:19:28.113779 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.113762    2574 scope.go:117] "RemoveContainer" containerID="c21c03ca3c6499ff43dfecd9c6cb8f6614f08b557136a71325506f4ee92fc23f"
Sep 14 03:19:28.114062 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:19:28.114043    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c21c03ca3c6499ff43dfecd9c6cb8f6614f08b557136a71325506f4ee92fc23f\": container with ID starting with c21c03ca3c6499ff43dfecd9c6cb8f6614f08b557136a71325506f4ee92fc23f not found: ID does not exist" containerID="c21c03ca3c6499ff43dfecd9c6cb8f6614f08b557136a71325506f4ee92fc23f"
Sep 14 03:19:28.114135 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.114067    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c21c03ca3c6499ff43dfecd9c6cb8f6614f08b557136a71325506f4ee92fc23f"} err="failed to get container status \"c21c03ca3c6499ff43dfecd9c6cb8f6614f08b557136a71325506f4ee92fc23f\": rpc error: code = NotFound desc = could not find container \"c21c03ca3c6499ff43dfecd9c6cb8f6614f08b557136a71325506f4ee92fc23f\": container with ID starting with c21c03ca3c6499ff43dfecd9c6cb8f6614f08b557136a71325506f4ee92fc23f not found: ID does not exist"
Sep 14 03:19:28.114135 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.114081    2574 scope.go:117] "RemoveContainer" containerID="cceff37ea00daba6c65dfb1e192feb049211425b5fa8bc97713bef38a0b00514"
Sep 14 03:19:28.114320 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:19:28.114303    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cceff37ea00daba6c65dfb1e192feb049211425b5fa8bc97713bef38a0b00514\": container with ID starting with cceff37ea00daba6c65dfb1e192feb049211425b5fa8bc97713bef38a0b00514 not found: ID does not exist" containerID="cceff37ea00daba6c65dfb1e192feb049211425b5fa8bc97713bef38a0b00514"
Sep 14 03:19:28.114361 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.114326    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cceff37ea00daba6c65dfb1e192feb049211425b5fa8bc97713bef38a0b00514"} err="failed to get container status \"cceff37ea00daba6c65dfb1e192feb049211425b5fa8bc97713bef38a0b00514\": rpc error: code = NotFound desc = could not find container \"cceff37ea00daba6c65dfb1e192feb049211425b5fa8bc97713bef38a0b00514\": container with ID starting with cceff37ea00daba6c65dfb1e192feb049211425b5fa8bc97713bef38a0b00514 not found: ID does not exist"
Sep 14 03:19:28.120875 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.120855    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"]
Sep 14 03:19:28.127761 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.127741    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s"]
Sep 14 03:19:28.216125 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:28.216072    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="752c7a53-2483-43b6-a723-30e32618e40c" path="/var/lib/kubelet/pods/752c7a53-2483-43b6-a723-30e32618e40c/volumes"
Sep 14 03:19:33.784664 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:33.784632    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7"]
Sep 14 03:19:33.788216 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:33.788190    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0c919e-bnft7"]
Sep 14 03:19:34.215301 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:34.215226    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6d75a77-cc0f-4a2f-8838-176c08b59f95" path="/var/lib/kubelet/pods/b6d75a77-cc0f-4a2f-8838-176c08b59f95/volumes"
Sep 14 03:19:36.735497 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:36.735457    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:19:41.181426 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.181394    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r"]
Sep 14 03:19:41.181796 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.181744    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="752c7a53-2483-43b6-a723-30e32618e40c" containerName="istio-proxy"
Sep 14 03:19:41.181796 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.181755    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="752c7a53-2483-43b6-a723-30e32618e40c" containerName="istio-proxy"
Sep 14 03:19:41.181796 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.181769    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6d75a77-cc0f-4a2f-8838-176c08b59f95" containerName="revoke-keys"
Sep 14 03:19:41.181796 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.181774    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6d75a77-cc0f-4a2f-8838-176c08b59f95" containerName="revoke-keys"
Sep 14 03:19:41.181930 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.181838    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="752c7a53-2483-43b6-a723-30e32618e40c" containerName="istio-proxy"
Sep 14 03:19:41.181930 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.181847    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="752c7a53-2483-43b6-a723-30e32618e40c" containerName="istio-proxy"
Sep 14 03:19:41.181930 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.181853    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6d75a77-cc0f-4a2f-8838-176c08b59f95" containerName="revoke-keys"
Sep 14 03:19:41.186205 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.186186    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r"
Sep 14 03:19:41.188944 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.188922    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4hf5p\""
Sep 14 03:19:41.189042 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.188922    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:19:41.194857 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.194837    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r"]
Sep 14 03:19:41.219246 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.219224    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28f558d0-2493-468a-adab-ca6c351108a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r\" (UID: \"28f558d0-2493-468a-adab-ca6c351108a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r"
Sep 14 03:19:41.320491 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.320456    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28f558d0-2493-468a-adab-ca6c351108a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r\" (UID: \"28f558d0-2493-468a-adab-ca6c351108a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r"
Sep 14 03:19:41.321046 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.321030    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28f558d0-2493-468a-adab-ca6c351108a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r\" (UID: \"28f558d0-2493-468a-adab-ca6c351108a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r"
Sep 14 03:19:41.497700 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.497678    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r"
Sep 14 03:19:41.769170 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:41.769148    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r"]
Sep 14 03:19:41.770985 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:19:41.770953    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f558d0_2493_468a_adab_ca6c351108a4.slice/crio-78b4fbaa5839539767193a74e59629cc54c20dee8b6667a6033201727635064c WatchSource:0}: Error finding container 78b4fbaa5839539767193a74e59629cc54c20dee8b6667a6033201727635064c: Status 404 returned error can't find the container with id 78b4fbaa5839539767193a74e59629cc54c20dee8b6667a6033201727635064c
Sep 14 03:19:42.170502 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:42.170423    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r" event={"ID":"28f558d0-2493-468a-adab-ca6c351108a4","Type":"ContainerStarted","Data":"87cf03e3529d31aa1d2d40794861ef302cfbc1b02ef3c0cf01a9a983cd0cf405"}
Sep 14 03:19:42.170502 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:42.170488    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r" event={"ID":"28f558d0-2493-468a-adab-ca6c351108a4","Type":"ContainerStarted","Data":"78b4fbaa5839539767193a74e59629cc54c20dee8b6667a6033201727635064c"}
Sep 14 03:19:42.189084 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:42.189040    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r" podStartSLOduration=1.189025505 podStartE2EDuration="1.189025505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:19:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:19:42.187077261 +0000 UTC m=+1728.500101080" watchObservedRunningTime="2026-09-14 03:19:42.189025505 +0000 UTC m=+1728.502049352"
Sep 14 03:19:43.177341 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:43.177304    2574 generic.go:356] "Generic (PLEG): container finished" podID="28f558d0-2493-468a-adab-ca6c351108a4" containerID="87cf03e3529d31aa1d2d40794861ef302cfbc1b02ef3c0cf01a9a983cd0cf405" exitCode=0
Sep 14 03:19:43.177547 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:43.177347    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r" event={"ID":"28f558d0-2493-468a-adab-ca6c351108a4","Type":"ContainerDied","Data":"87cf03e3529d31aa1d2d40794861ef302cfbc1b02ef3c0cf01a9a983cd0cf405"}
Sep 14 03:19:44.314075 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:44.314051    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r"
Sep 14 03:19:44.343460 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:44.343439    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28f558d0-2493-468a-adab-ca6c351108a4-maas-api-service-ca\") pod \"28f558d0-2493-468a-adab-ca6c351108a4\" (UID: \"28f558d0-2493-468a-adab-ca6c351108a4\") "
Sep 14 03:19:44.343812 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:44.343792    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28f558d0-2493-468a-adab-ca6c351108a4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "28f558d0-2493-468a-adab-ca6c351108a4" (UID: "28f558d0-2493-468a-adab-ca6c351108a4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:19:44.445012 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:44.444948    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28f558d0-2493-468a-adab-ca6c351108a4-maas-api-service-ca\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:19:45.189015 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:45.188977    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r" event={"ID":"28f558d0-2493-468a-adab-ca6c351108a4","Type":"ContainerDied","Data":"78b4fbaa5839539767193a74e59629cc54c20dee8b6667a6033201727635064c"}
Sep 14 03:19:45.189015 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:45.188997    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r"
Sep 14 03:19:45.189239 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:45.189028    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78b4fbaa5839539767193a74e59629cc54c20dee8b6667a6033201727635064c"
Sep 14 03:19:51.640314 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:51.640283    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f83c87-29822595-xbb48"]
Sep 14 03:19:51.643913 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:51.643885    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-f83c87-29822595-xbb48"]
Sep 14 03:19:51.872247 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:51.872219    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r"]
Sep 14 03:19:51.875527 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:51.875490    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f83c87-tzn9r"]
Sep 14 03:19:52.215877 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:52.215843    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28f558d0-2493-468a-adab-ca6c351108a4" path="/var/lib/kubelet/pods/28f558d0-2493-468a-adab-ca6c351108a4/volumes"
Sep 14 03:19:52.216204 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:52.216183    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8877cfcf-9755-421f-8607-d9abed3690a1" path="/var/lib/kubelet/pods/8877cfcf-9755-421f-8607-d9abed3690a1/volumes"
Sep 14 03:19:54.491604 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:54.491564    2574 scope.go:117] "RemoveContainer" containerID="7abfe487d89677439d3751f3f8f55526e1ebe1c301af57aafefeb661365af44e"
Sep 14 03:19:54.547872 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:54.547845    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:19:58.881708 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:58.881675    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"]
Sep 14 03:19:58.882121 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:19:58.881864    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9" podUID="aa86f69b-8639-4711-a69b-4872e6f56368" containerName="istio-proxy" containerID="cri-o://ace602a9e6cf8f7d40899ec64bb26fbe9979a8a34ac379edfa0001ec0e0278cc" gracePeriod=30
Sep 14 03:20:02.660988 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:02.660947    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:20:02.695433 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:02.695396    2574 patch_prober.go:28] interesting pod/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 03:20:02.695599 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:02.695462    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9" podUID="aa86f69b-8639-4711-a69b-4872e6f56368" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 03:20:04.130983 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.130961    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:20:04.201801 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.201746    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-istio-envoy\") pod \"aa86f69b-8639-4711-a69b-4872e6f56368\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") "
Sep 14 03:20:04.201801 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.201789    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-workload-certs\") pod \"aa86f69b-8639-4711-a69b-4872e6f56368\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") "
Sep 14 03:20:04.201955 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.201806    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-istio-data\") pod \"aa86f69b-8639-4711-a69b-4872e6f56368\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") "
Sep 14 03:20:04.201955 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.201834    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-workload-socket\") pod \"aa86f69b-8639-4711-a69b-4872e6f56368\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") "
Sep 14 03:20:04.201955 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.201854    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dr6nl\" (UniqueName: \"kubernetes.io/projected/aa86f69b-8639-4711-a69b-4872e6f56368-kube-api-access-dr6nl\") pod \"aa86f69b-8639-4711-a69b-4872e6f56368\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") "
Sep 14 03:20:04.201955 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.201876    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa86f69b-8639-4711-a69b-4872e6f56368-istio-podinfo\") pod \"aa86f69b-8639-4711-a69b-4872e6f56368\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") "
Sep 14 03:20:04.201955 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.201906    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa86f69b-8639-4711-a69b-4872e6f56368-istiod-ca-cert\") pod \"aa86f69b-8639-4711-a69b-4872e6f56368\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") "
Sep 14 03:20:04.201955 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.201929    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-credential-socket\") pod \"aa86f69b-8639-4711-a69b-4872e6f56368\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") "
Sep 14 03:20:04.201955 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.201954    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa86f69b-8639-4711-a69b-4872e6f56368-istio-token\") pod \"aa86f69b-8639-4711-a69b-4872e6f56368\" (UID: \"aa86f69b-8639-4711-a69b-4872e6f56368\") "
Sep 14 03:20:04.202322 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.202054    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aa86f69b-8639-4711-a69b-4872e6f56368" (UID: "aa86f69b-8639-4711-a69b-4872e6f56368"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:20:04.202322 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.202156    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-istio-data" (OuterVolumeSpecName: "istio-data") pod "aa86f69b-8639-4711-a69b-4872e6f56368" (UID: "aa86f69b-8639-4711-a69b-4872e6f56368"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:20:04.202322 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.202293    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-workload-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:20:04.202322 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.202312    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-istio-data\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:20:04.202322 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.202287    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aa86f69b-8639-4711-a69b-4872e6f56368" (UID: "aa86f69b-8639-4711-a69b-4872e6f56368"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:20:04.202688 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.202341    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aa86f69b-8639-4711-a69b-4872e6f56368" (UID: "aa86f69b-8639-4711-a69b-4872e6f56368"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:20:04.202688 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.202376    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa86f69b-8639-4711-a69b-4872e6f56368-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aa86f69b-8639-4711-a69b-4872e6f56368" (UID: "aa86f69b-8639-4711-a69b-4872e6f56368"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:20:04.204301 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.204274    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa86f69b-8639-4711-a69b-4872e6f56368-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aa86f69b-8639-4711-a69b-4872e6f56368" (UID: "aa86f69b-8639-4711-a69b-4872e6f56368"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:20:04.204421 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.204340    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa86f69b-8639-4711-a69b-4872e6f56368-istio-token" (OuterVolumeSpecName: "istio-token") pod "aa86f69b-8639-4711-a69b-4872e6f56368" (UID: "aa86f69b-8639-4711-a69b-4872e6f56368"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:20:04.204624 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.204608    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aa86f69b-8639-4711-a69b-4872e6f56368" (UID: "aa86f69b-8639-4711-a69b-4872e6f56368"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:20:04.204680 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.204618    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa86f69b-8639-4711-a69b-4872e6f56368-kube-api-access-dr6nl" (OuterVolumeSpecName: "kube-api-access-dr6nl") pod "aa86f69b-8639-4711-a69b-4872e6f56368" (UID: "aa86f69b-8639-4711-a69b-4872e6f56368"). InnerVolumeSpecName "kube-api-access-dr6nl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:20:04.281208 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.281178    2574 generic.go:356] "Generic (PLEG): container finished" podID="aa86f69b-8639-4711-a69b-4872e6f56368" containerID="ace602a9e6cf8f7d40899ec64bb26fbe9979a8a34ac379edfa0001ec0e0278cc" exitCode=0
Sep 14 03:20:04.281310 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.281224    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"
Sep 14 03:20:04.281310 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.281260    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9" event={"ID":"aa86f69b-8639-4711-a69b-4872e6f56368","Type":"ContainerDied","Data":"ace602a9e6cf8f7d40899ec64bb26fbe9979a8a34ac379edfa0001ec0e0278cc"}
Sep 14 03:20:04.281310 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.281308    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9" event={"ID":"aa86f69b-8639-4711-a69b-4872e6f56368","Type":"ContainerDied","Data":"7f6321a4f74d5a0417ad4521aa01c1a19e8cad372832a261fdb467a19f1c1fc3"}
Sep 14 03:20:04.281476 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.281330    2574 scope.go:117] "RemoveContainer" containerID="ace602a9e6cf8f7d40899ec64bb26fbe9979a8a34ac379edfa0001ec0e0278cc"
Sep 14 03:20:04.290140 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.290125    2574 scope.go:117] "RemoveContainer" containerID="f098b9b98f48b43716fc8847239330e587197e338e494791764233a479641a83"
Sep 14 03:20:04.298276 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.298259    2574 scope.go:117] "RemoveContainer" containerID="ace602a9e6cf8f7d40899ec64bb26fbe9979a8a34ac379edfa0001ec0e0278cc"
Sep 14 03:20:04.298599 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:20:04.298560    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ace602a9e6cf8f7d40899ec64bb26fbe9979a8a34ac379edfa0001ec0e0278cc\": container with ID starting with ace602a9e6cf8f7d40899ec64bb26fbe9979a8a34ac379edfa0001ec0e0278cc not found: ID does not exist" containerID="ace602a9e6cf8f7d40899ec64bb26fbe9979a8a34ac379edfa0001ec0e0278cc"
Sep 14 03:20:04.298599 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.298588    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ace602a9e6cf8f7d40899ec64bb26fbe9979a8a34ac379edfa0001ec0e0278cc"} err="failed to get container status \"ace602a9e6cf8f7d40899ec64bb26fbe9979a8a34ac379edfa0001ec0e0278cc\": rpc error: code = NotFound desc = could not find container \"ace602a9e6cf8f7d40899ec64bb26fbe9979a8a34ac379edfa0001ec0e0278cc\": container with ID starting with ace602a9e6cf8f7d40899ec64bb26fbe9979a8a34ac379edfa0001ec0e0278cc not found: ID does not exist"
Sep 14 03:20:04.298841 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.298611    2574 scope.go:117] "RemoveContainer" containerID="f098b9b98f48b43716fc8847239330e587197e338e494791764233a479641a83"
Sep 14 03:20:04.299029 ip-10-0-133-188 kubenswrapper[2574]: E0914 03:20:04.299000    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f098b9b98f48b43716fc8847239330e587197e338e494791764233a479641a83\": container with ID starting with f098b9b98f48b43716fc8847239330e587197e338e494791764233a479641a83 not found: ID does not exist" containerID="f098b9b98f48b43716fc8847239330e587197e338e494791764233a479641a83"
Sep 14 03:20:04.299121 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.299038    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f098b9b98f48b43716fc8847239330e587197e338e494791764233a479641a83"} err="failed to get container status \"f098b9b98f48b43716fc8847239330e587197e338e494791764233a479641a83\": rpc error: code = NotFound desc = could not find container \"f098b9b98f48b43716fc8847239330e587197e338e494791764233a479641a83\": container with ID starting with f098b9b98f48b43716fc8847239330e587197e338e494791764233a479641a83 not found: ID does not exist"
Sep 14 03:20:04.301077 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.301056    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"]
Sep 14 03:20:04.303603 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.303581    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-istio-envoy\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:20:04.303711 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.303609    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-workload-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:20:04.303711 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.303626    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dr6nl\" (UniqueName: \"kubernetes.io/projected/aa86f69b-8639-4711-a69b-4872e6f56368-kube-api-access-dr6nl\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:20:04.303711 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.303640    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa86f69b-8639-4711-a69b-4872e6f56368-istio-podinfo\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:20:04.303711 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.303654    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa86f69b-8639-4711-a69b-4872e6f56368-istiod-ca-cert\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:20:04.303711 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.303664    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa86f69b-8639-4711-a69b-4872e6f56368-credential-socket\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:20:04.303711 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.303672    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa86f69b-8639-4711-a69b-4872e6f56368-istio-token\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:20:04.305413 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:04.305395    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9"]
Sep 14 03:20:06.215927 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:06.215894    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa86f69b-8639-4711-a69b-4872e6f56368" path="/var/lib/kubelet/pods/aa86f69b-8639-4711-a69b-4872e6f56368/volumes"
Sep 14 03:20:54.251729 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:54.251703    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 03:20:54.258300 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:54.258279    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 03:20:54.312977 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:54.312954    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 03:20:54.319505 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:20:54.319482    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 03:21:24.160046 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:21:24.160013    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:21:52.043831 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:21:52.043793    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:21:54.537286 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:21:54.537252    2574 scope.go:117] "RemoveContainer" containerID="63c80d54c9e53eca60f89ccf2c0d6051c52893f82210490862ab120a8d36ab66"
Sep 14 03:21:54.547327 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:21:54.547298    2574 scope.go:117] "RemoveContainer" containerID="fd7d7ff002fed781a2a73149279eceae83f4875a274b4b0279c58b91335e554a"
Sep 14 03:21:59.048307 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:21:59.048265    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:22:05.069391 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:22:05.069354    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:22:13.236788 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:22:13.236755    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:22:27.234026 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:22:27.233937    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:22:35.051538 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:22:35.051484    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:22:54.568695 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:22:54.568662    2574 scope.go:117] "RemoveContainer" containerID="344972224ea75ff484bca2413f2089f8f9e9a564ff9bcdfed8cd5e887111b2de"
Sep 14 03:23:54.587575 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:23:54.587542    2574 scope.go:117] "RemoveContainer" containerID="8fbef5bdbf2e095dc10326339ecede3d74d924c5b35b64052f8a4588186aeef1"
Sep 14 03:24:02.942722 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:02.942692    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:24:10.638660 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:10.638624    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:24:16.232761 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:16.232727    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:24:24.586601 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.586568    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"]
Sep 14 03:24:24.587099 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.587052    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28f558d0-2493-468a-adab-ca6c351108a4" containerName="revoke-keys"
Sep 14 03:24:24.587099 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.587070    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="28f558d0-2493-468a-adab-ca6c351108a4" containerName="revoke-keys"
Sep 14 03:24:24.587099 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.587083    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="752c7a53-2483-43b6-a723-30e32618e40c" containerName="istio-proxy"
Sep 14 03:24:24.587099 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.587090    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="752c7a53-2483-43b6-a723-30e32618e40c" containerName="istio-proxy"
Sep 14 03:24:24.587315 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.587109    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa86f69b-8639-4711-a69b-4872e6f56368" containerName="istio-proxy"
Sep 14 03:24:24.587315 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.587117    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa86f69b-8639-4711-a69b-4872e6f56368" containerName="istio-proxy"
Sep 14 03:24:24.587315 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.587153    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa86f69b-8639-4711-a69b-4872e6f56368" containerName="istio-proxy"
Sep 14 03:24:24.587315 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.587162    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa86f69b-8639-4711-a69b-4872e6f56368" containerName="istio-proxy"
Sep 14 03:24:24.587315 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.587243    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa86f69b-8639-4711-a69b-4872e6f56368" containerName="istio-proxy"
Sep 14 03:24:24.587315 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.587257    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="28f558d0-2493-468a-adab-ca6c351108a4" containerName="revoke-keys"
Sep 14 03:24:24.587315 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.587268    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa86f69b-8639-4711-a69b-4872e6f56368" containerName="istio-proxy"
Sep 14 03:24:24.590300 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.590280    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.593243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.593220    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-647b49d0-kserve-self-signed-certs\""
Sep 14 03:24:24.602448 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.602424    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"]
Sep 14 03:24:24.746144 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.746120    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e1f2527-6ed1-4868-bd46-dab954468490-tls-certs\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.746240 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.746152    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-dshm\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.746240 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.746213    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-tmp-dir\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.746240 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.746232    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-home\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.746377 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.746252    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-model-cache\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.746377 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.746277    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqztn\" (UniqueName: \"kubernetes.io/projected/8e1f2527-6ed1-4868-bd46-dab954468490-kube-api-access-gqztn\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.847388 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.847319    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e1f2527-6ed1-4868-bd46-dab954468490-tls-certs\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.847388 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.847350    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-dshm\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.847603 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.847399    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-tmp-dir\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.847603 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.847420    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-home\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.847603 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.847441    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-model-cache\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.847603 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.847470    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqztn\" (UniqueName: \"kubernetes.io/projected/8e1f2527-6ed1-4868-bd46-dab954468490-kube-api-access-gqztn\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.847945 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.847924    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-model-cache\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.848010 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.847950    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-home\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.848010 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.847995    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-tmp-dir\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.850478 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.850451    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-dshm\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.851130 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.851108    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e1f2527-6ed1-4868-bd46-dab954468490-tls-certs\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.857484 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.857457    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqztn\" (UniqueName: \"kubernetes.io/projected/8e1f2527-6ed1-4868-bd46-dab954468490-kube-api-access-gqztn\") pod \"e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") " pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:24.899564 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:24.899539    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:25.174380 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:25.174306    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"]
Sep 14 03:24:25.177157 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:24:25.177128    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e1f2527_6ed1_4868_bd46_dab954468490.slice/crio-890713decec1dcdf61c1ff8536326c8aeec497b0d2861b5fcffd7ea52752ebc3 WatchSource:0}: Error finding container 890713decec1dcdf61c1ff8536326c8aeec497b0d2861b5fcffd7ea52752ebc3: Status 404 returned error can't find the container with id 890713decec1dcdf61c1ff8536326c8aeec497b0d2861b5fcffd7ea52752ebc3
Sep 14 03:24:25.554788 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:25.554732    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn" event={"ID":"8e1f2527-6ed1-4868-bd46-dab954468490","Type":"ContainerStarted","Data":"4683d007f856a04d9763770351c389f29ce3a0ab0af0e61fa6ee96c7592c3a3e"}
Sep 14 03:24:25.554788 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:25.554789    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn" event={"ID":"8e1f2527-6ed1-4868-bd46-dab954468490","Type":"ContainerStarted","Data":"890713decec1dcdf61c1ff8536326c8aeec497b0d2861b5fcffd7ea52752ebc3"}
Sep 14 03:24:25.572485 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:25.572349    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn" podStartSLOduration=1.572332937 podStartE2EDuration="1.572332937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:24:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:24:25.571993869 +0000 UTC m=+2011.885017689" watchObservedRunningTime="2026-09-14 03:24:25.572332937 +0000 UTC m=+2011.885356746"
Sep 14 03:24:27.142750 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:27.142708    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:24:34.900613 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:34.900575    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:34.901110 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:34.900637    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:34.913208 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:34.913179    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:35.612169 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:35.612138    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:24:41.615177 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:41.615144    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:24:52.596324 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:52.596291    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"]
Sep 14 03:24:54.830058 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:24:54.830027    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:25:03.550639 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:03.550596    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:25:07.615388 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.615347    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn" podUID="8e1f2527-6ed1-4868-bd46-dab954468490" containerName="main" containerID="cri-o://4683d007f856a04d9763770351c389f29ce3a0ab0af0e61fa6ee96c7592c3a3e" gracePeriod=45
Sep 14 03:25:07.763660 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.763632    2574 generic.go:356] "Generic (PLEG): container finished" podID="8e1f2527-6ed1-4868-bd46-dab954468490" containerID="4683d007f856a04d9763770351c389f29ce3a0ab0af0e61fa6ee96c7592c3a3e" exitCode=0
Sep 14 03:25:07.763829 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.763670    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn" event={"ID":"8e1f2527-6ed1-4868-bd46-dab954468490","Type":"ContainerDied","Data":"4683d007f856a04d9763770351c389f29ce3a0ab0af0e61fa6ee96c7592c3a3e"}
Sep 14 03:25:07.862095 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.862072    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:25:07.970243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.970179    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqztn\" (UniqueName: \"kubernetes.io/projected/8e1f2527-6ed1-4868-bd46-dab954468490-kube-api-access-gqztn\") pod \"8e1f2527-6ed1-4868-bd46-dab954468490\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") "
Sep 14 03:25:07.970243 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.970224    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-dshm\") pod \"8e1f2527-6ed1-4868-bd46-dab954468490\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") "
Sep 14 03:25:07.970423 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.970256    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e1f2527-6ed1-4868-bd46-dab954468490-tls-certs\") pod \"8e1f2527-6ed1-4868-bd46-dab954468490\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") "
Sep 14 03:25:07.970423 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.970286    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-model-cache\") pod \"8e1f2527-6ed1-4868-bd46-dab954468490\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") "
Sep 14 03:25:07.970423 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.970312    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-home\") pod \"8e1f2527-6ed1-4868-bd46-dab954468490\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") "
Sep 14 03:25:07.970423 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.970355    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-tmp-dir\") pod \"8e1f2527-6ed1-4868-bd46-dab954468490\" (UID: \"8e1f2527-6ed1-4868-bd46-dab954468490\") "
Sep 14 03:25:07.970649 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.970581    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-model-cache" (OuterVolumeSpecName: "model-cache") pod "8e1f2527-6ed1-4868-bd46-dab954468490" (UID: "8e1f2527-6ed1-4868-bd46-dab954468490"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:25:07.970649 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.970596    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-home" (OuterVolumeSpecName: "home") pod "8e1f2527-6ed1-4868-bd46-dab954468490" (UID: "8e1f2527-6ed1-4868-bd46-dab954468490"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:25:07.970764 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.970710    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-model-cache\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:25:07.970764 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.970729    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-home\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:25:07.970764 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.970752    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8e1f2527-6ed1-4868-bd46-dab954468490" (UID: "8e1f2527-6ed1-4868-bd46-dab954468490"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:25:07.972640 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.972618    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e1f2527-6ed1-4868-bd46-dab954468490-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8e1f2527-6ed1-4868-bd46-dab954468490" (UID: "8e1f2527-6ed1-4868-bd46-dab954468490"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:25:07.972741 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.972676    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-dshm" (OuterVolumeSpecName: "dshm") pod "8e1f2527-6ed1-4868-bd46-dab954468490" (UID: "8e1f2527-6ed1-4868-bd46-dab954468490"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:25:07.972741 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:07.972685    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e1f2527-6ed1-4868-bd46-dab954468490-kube-api-access-gqztn" (OuterVolumeSpecName: "kube-api-access-gqztn") pod "8e1f2527-6ed1-4868-bd46-dab954468490" (UID: "8e1f2527-6ed1-4868-bd46-dab954468490"). InnerVolumeSpecName "kube-api-access-gqztn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:25:08.073882 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:08.073855    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqztn\" (UniqueName: \"kubernetes.io/projected/8e1f2527-6ed1-4868-bd46-dab954468490-kube-api-access-gqztn\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:25:08.074068 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:08.074049    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-dshm\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:25:08.074275 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:08.074257    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e1f2527-6ed1-4868-bd46-dab954468490-tls-certs\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:25:08.074335 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:08.074280    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e1f2527-6ed1-4868-bd46-dab954468490-tmp-dir\") on node \"ip-10-0-133-188.ec2.internal\" DevicePath \"\""
Sep 14 03:25:08.769854 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:08.769818    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn" event={"ID":"8e1f2527-6ed1-4868-bd46-dab954468490","Type":"ContainerDied","Data":"890713decec1dcdf61c1ff8536326c8aeec497b0d2861b5fcffd7ea52752ebc3"}
Sep 14 03:25:08.770319 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:08.769861    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"
Sep 14 03:25:08.770319 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:08.769874    2574 scope.go:117] "RemoveContainer" containerID="4683d007f856a04d9763770351c389f29ce3a0ab0af0e61fa6ee96c7592c3a3e"
Sep 14 03:25:08.787431 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:08.787409    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"]
Sep 14 03:25:08.791868 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:08.791847    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-647b49d0-kserve-7f78966547-wn8vn"]
Sep 14 03:25:10.214688 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:10.214653    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e1f2527-6ed1-4868-bd46-dab954468490" path="/var/lib/kubelet/pods/8e1f2527-6ed1-4868-bd46-dab954468490/volumes"
Sep 14 03:25:10.602525 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:10.602480    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:25:17.158832 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:17.158742    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:25:25.070599 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.070563    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:25:25.516212 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.516178    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"]
Sep 14 03:25:25.516500 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.516489    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e1f2527-6ed1-4868-bd46-dab954468490" containerName="main"
Sep 14 03:25:25.516567 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.516502    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e1f2527-6ed1-4868-bd46-dab954468490" containerName="main"
Sep 14 03:25:25.516605 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.516568    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e1f2527-6ed1-4868-bd46-dab954468490" containerName="main"
Sep 14 03:25:25.520451 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.520426    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:25.523218 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.523186    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:25:25.524353 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.524333    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 03:25:25.524463 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.524405    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vckj5\""
Sep 14 03:25:25.524547 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.524476    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 03:25:25.524690 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.524669    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 14 03:25:25.533633 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.533614    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"]
Sep 14 03:25:25.610976 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.610941    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a79052af-308f-494a-a45a-c8bc5b3445d8-metrics-certs\") pod \"maas-api-86d9ddc875-pkrp5\" (UID: \"a79052af-308f-494a-a45a-c8bc5b3445d8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:25.611174 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.611030    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a79052af-308f-494a-a45a-c8bc5b3445d8-maas-api-tls\") pod \"maas-api-86d9ddc875-pkrp5\" (UID: \"a79052af-308f-494a-a45a-c8bc5b3445d8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:25.611174 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.611055    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7pk8\" (UniqueName: \"kubernetes.io/projected/a79052af-308f-494a-a45a-c8bc5b3445d8-kube-api-access-t7pk8\") pod \"maas-api-86d9ddc875-pkrp5\" (UID: \"a79052af-308f-494a-a45a-c8bc5b3445d8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:25.611174 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.611086    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a79052af-308f-494a-a45a-c8bc5b3445d8-openshift-service-ca\") pod \"maas-api-86d9ddc875-pkrp5\" (UID: \"a79052af-308f-494a-a45a-c8bc5b3445d8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:25.712349 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.712311    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a79052af-308f-494a-a45a-c8bc5b3445d8-metrics-certs\") pod \"maas-api-86d9ddc875-pkrp5\" (UID: \"a79052af-308f-494a-a45a-c8bc5b3445d8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:25.712562 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.712370    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a79052af-308f-494a-a45a-c8bc5b3445d8-maas-api-tls\") pod \"maas-api-86d9ddc875-pkrp5\" (UID: \"a79052af-308f-494a-a45a-c8bc5b3445d8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:25.712562 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.712386    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7pk8\" (UniqueName: \"kubernetes.io/projected/a79052af-308f-494a-a45a-c8bc5b3445d8-kube-api-access-t7pk8\") pod \"maas-api-86d9ddc875-pkrp5\" (UID: \"a79052af-308f-494a-a45a-c8bc5b3445d8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:25.712562 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.712503    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a79052af-308f-494a-a45a-c8bc5b3445d8-openshift-service-ca\") pod \"maas-api-86d9ddc875-pkrp5\" (UID: \"a79052af-308f-494a-a45a-c8bc5b3445d8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:25.713313 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.713286    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a79052af-308f-494a-a45a-c8bc5b3445d8-openshift-service-ca\") pod \"maas-api-86d9ddc875-pkrp5\" (UID: \"a79052af-308f-494a-a45a-c8bc5b3445d8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:25.715097 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.715070    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a79052af-308f-494a-a45a-c8bc5b3445d8-maas-api-tls\") pod \"maas-api-86d9ddc875-pkrp5\" (UID: \"a79052af-308f-494a-a45a-c8bc5b3445d8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:25.715186 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.715131    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a79052af-308f-494a-a45a-c8bc5b3445d8-metrics-certs\") pod \"maas-api-86d9ddc875-pkrp5\" (UID: \"a79052af-308f-494a-a45a-c8bc5b3445d8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:25.721788 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.721747    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7pk8\" (UniqueName: \"kubernetes.io/projected/a79052af-308f-494a-a45a-c8bc5b3445d8-kube-api-access-t7pk8\") pod \"maas-api-86d9ddc875-pkrp5\" (UID: \"a79052af-308f-494a-a45a-c8bc5b3445d8\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:25.830670 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:25.830616    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:26.115242 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:26.115213    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"]
Sep 14 03:25:26.118717 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:25:26.118685    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda79052af_308f_494a_a45a_c8bc5b3445d8.slice/crio-9d07f50190595b5369fe351d21b02c38a6bd629b75dabf2e20b3ef863b9ed725 WatchSource:0}: Error finding container 9d07f50190595b5369fe351d21b02c38a6bd629b75dabf2e20b3ef863b9ed725: Status 404 returned error can't find the container with id 9d07f50190595b5369fe351d21b02c38a6bd629b75dabf2e20b3ef863b9ed725
Sep 14 03:25:26.120646 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:26.120623    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 03:25:26.860666 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:26.860620    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5" event={"ID":"a79052af-308f-494a-a45a-c8bc5b3445d8","Type":"ContainerStarted","Data":"9d07f50190595b5369fe351d21b02c38a6bd629b75dabf2e20b3ef863b9ed725"}
Sep 14 03:25:28.872049 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:28.872004    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5" event={"ID":"a79052af-308f-494a-a45a-c8bc5b3445d8","Type":"ContainerStarted","Data":"d0ae62b66bbf8bae9bf25aff5b50f917869d150a5c619dfd191e526c99904fe4"}
Sep 14 03:25:28.872411 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:28.872112    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:28.894336 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:28.894285    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5" podStartSLOduration=1.6482720450000001 podStartE2EDuration="3.894268127s" totalImagesPullingTime="2.245996082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:25:28.894003196 +0000 UTC m=+2075.207027017" watchObservedRunningTime="2026-09-14 03:25:28.894268127 +0000 UTC m=+2075.207291947"
Sep 14 03:25:31.845865 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:31.845834    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:25:34.882329 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:34.882299    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-pkrp5"
Sep 14 03:25:37.645412 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:37.645374    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:25:44.283983 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:44.283949    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-p2cbh"]
Sep 14 03:25:50.742224 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:50.742195    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wdp4n_2a5808dc-86c4-4507-8281-09e0c2910f77/global-pull-secret-syncer/0.log"
Sep 14 03:25:50.826583 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:50.826549    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nsgwq_1a9bb9b8-0a23-445a-9074-930df7d80ea0/konnectivity-agent/0.log"
Sep 14 03:25:50.908911 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:50.908889    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-188.ec2.internal_07e0d660232ee4077dd70b8b3442d37b/haproxy/0.log"
Sep 14 03:25:54.279097 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:54.279073    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 03:25:54.285139 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:54.285116    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 03:25:54.341707 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:54.341684    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 03:25:54.347656 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:54.347636    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4jxp_6608dc19-84cb-4438-8940-38f32c25381c/ovn-acl-logging/0.log"
Sep 14 03:25:54.613661 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:54.613590    2574 scope.go:117] "RemoveContainer" containerID="34337d18db9393109b95af5499b3e18f6e332d345247bc46a702e3ebbc36bc8f"
Sep 14 03:25:54.621871 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:54.621851    2574 scope.go:117] "RemoveContainer" containerID="87cf03e3529d31aa1d2d40794861ef302cfbc1b02ef3c0cf01a9a983cd0cf405"
Sep 14 03:25:55.532473 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:55.532446    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-tj9tz_5e0482aa-e17e-4624-8aa2-c9d6f2bf4e70/manager/0.log"
Sep 14 03:25:55.596690 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:55.596661    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-k7gcc_bd6b9db2-2a69-4d03-b442-ba6308955f3f/kuadrant-console-plugin/0.log"
Sep 14 03:25:55.657789 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:55.657758    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-p2cbh_cfbd2a72-d324-456f-be02-832d3a442bdb/limitador/0.log"
Sep 14 03:25:57.377693 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:57.377668    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-fkpqf_ff4eb2f8-e574-4e2c-a59f-0c715f466adb/cluster-monitoring-operator/0.log"
Sep 14 03:25:57.554707 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:57.554680    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cbwjn_d0141b5c-8ece-4ae7-bf48-b756da7cf4c8/node-exporter/0.log"
Sep 14 03:25:57.577059 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:57.577040    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cbwjn_d0141b5c-8ece-4ae7-bf48-b756da7cf4c8/kube-rbac-proxy/0.log"
Sep 14 03:25:57.600210 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:25:57.600189    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cbwjn_d0141b5c-8ece-4ae7-bf48-b756da7cf4c8/init-textfile/0.log"
Sep 14 03:26:00.633846 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:00.633811    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/1.log"
Sep 14 03:26:00.638434 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:00.638413    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-lnp2z_8e2a325e-9309-46ae-8fca-2cafce3df418/console-operator/2.log"
Sep 14 03:26:00.889427 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:00.889359    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"]
Sep 14 03:26:00.893446 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:00.893431    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:00.896242 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:00.896222    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9xgtv\"/\"openshift-service-ca.crt\""
Sep 14 03:26:00.897435 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:00.897413    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9xgtv\"/\"default-dockercfg-dfvsv\""
Sep 14 03:26:00.897435 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:00.897433    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9xgtv\"/\"kube-root-ca.crt\""
Sep 14 03:26:00.901342 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:00.901321    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"]
Sep 14 03:26:00.994107 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:00.994080    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/08813945-3a44-43d9-983a-5c80630b24ec-proc\") pod \"perf-node-gather-daemonset-tmwf7\" (UID: \"08813945-3a44-43d9-983a-5c80630b24ec\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:00.994226 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:00.994120    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08813945-3a44-43d9-983a-5c80630b24ec-sys\") pod \"perf-node-gather-daemonset-tmwf7\" (UID: \"08813945-3a44-43d9-983a-5c80630b24ec\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:00.994226 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:00.994143    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08813945-3a44-43d9-983a-5c80630b24ec-lib-modules\") pod \"perf-node-gather-daemonset-tmwf7\" (UID: \"08813945-3a44-43d9-983a-5c80630b24ec\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:00.994354 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:00.994248    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkdjt\" (UniqueName: \"kubernetes.io/projected/08813945-3a44-43d9-983a-5c80630b24ec-kube-api-access-kkdjt\") pod \"perf-node-gather-daemonset-tmwf7\" (UID: \"08813945-3a44-43d9-983a-5c80630b24ec\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:00.994354 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:00.994284    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/08813945-3a44-43d9-983a-5c80630b24ec-podres\") pod \"perf-node-gather-daemonset-tmwf7\" (UID: \"08813945-3a44-43d9-983a-5c80630b24ec\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:01.095435 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:01.095402    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08813945-3a44-43d9-983a-5c80630b24ec-sys\") pod \"perf-node-gather-daemonset-tmwf7\" (UID: \"08813945-3a44-43d9-983a-5c80630b24ec\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:01.095435 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:01.095439    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08813945-3a44-43d9-983a-5c80630b24ec-lib-modules\") pod \"perf-node-gather-daemonset-tmwf7\" (UID: \"08813945-3a44-43d9-983a-5c80630b24ec\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:01.095696 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:01.095466    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkdjt\" (UniqueName: \"kubernetes.io/projected/08813945-3a44-43d9-983a-5c80630b24ec-kube-api-access-kkdjt\") pod \"perf-node-gather-daemonset-tmwf7\" (UID: \"08813945-3a44-43d9-983a-5c80630b24ec\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:01.095696 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:01.095485    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/08813945-3a44-43d9-983a-5c80630b24ec-podres\") pod \"perf-node-gather-daemonset-tmwf7\" (UID: \"08813945-3a44-43d9-983a-5c80630b24ec\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:01.095696 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:01.095542    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08813945-3a44-43d9-983a-5c80630b24ec-sys\") pod \"perf-node-gather-daemonset-tmwf7\" (UID: \"08813945-3a44-43d9-983a-5c80630b24ec\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:01.095696 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:01.095609    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08813945-3a44-43d9-983a-5c80630b24ec-lib-modules\") pod \"perf-node-gather-daemonset-tmwf7\" (UID: \"08813945-3a44-43d9-983a-5c80630b24ec\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:01.095696 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:01.095628    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/08813945-3a44-43d9-983a-5c80630b24ec-podres\") pod \"perf-node-gather-daemonset-tmwf7\" (UID: \"08813945-3a44-43d9-983a-5c80630b24ec\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:01.095696 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:01.095611    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/08813945-3a44-43d9-983a-5c80630b24ec-proc\") pod \"perf-node-gather-daemonset-tmwf7\" (UID: \"08813945-3a44-43d9-983a-5c80630b24ec\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:01.095696 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:01.095649    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/08813945-3a44-43d9-983a-5c80630b24ec-proc\") pod \"perf-node-gather-daemonset-tmwf7\" (UID: \"08813945-3a44-43d9-983a-5c80630b24ec\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:01.104478 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:01.104454    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkdjt\" (UniqueName: \"kubernetes.io/projected/08813945-3a44-43d9-983a-5c80630b24ec-kube-api-access-kkdjt\") pod \"perf-node-gather-daemonset-tmwf7\" (UID: \"08813945-3a44-43d9-983a-5c80630b24ec\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:01.204648 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:01.204585    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:01.474035 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:01.474007    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"]
Sep 14 03:26:01.476475 ip-10-0-133-188 kubenswrapper[2574]: W0914 03:26:01.476439    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod08813945_3a44_43d9_983a_5c80630b24ec.slice/crio-049751e105a383ac78e0017a654d622fe5f59a58dbe31c525e5a1b2fc93623d6 WatchSource:0}: Error finding container 049751e105a383ac78e0017a654d622fe5f59a58dbe31c525e5a1b2fc93623d6: Status 404 returned error can't find the container with id 049751e105a383ac78e0017a654d622fe5f59a58dbe31c525e5a1b2fc93623d6
Sep 14 03:26:01.994375 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:01.994347    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-mrh5b_8e237d4c-dde5-4191-84f6-e8e32ff7d5d8/volume-data-source-validator/0.log"
Sep 14 03:26:02.037564 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:02.037503    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7" event={"ID":"08813945-3a44-43d9-983a-5c80630b24ec","Type":"ContainerStarted","Data":"a3b0a756e99fb58cf006ade525dff7275f69d7b3fc53a633a3bbc8734cbf987c"}
Sep 14 03:26:02.037740 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:02.037592    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7" event={"ID":"08813945-3a44-43d9-983a-5c80630b24ec","Type":"ContainerStarted","Data":"049751e105a383ac78e0017a654d622fe5f59a58dbe31c525e5a1b2fc93623d6"}
Sep 14 03:26:02.037740 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:02.037639    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:02.064607 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:02.064560    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7" podStartSLOduration=2.064545553 podStartE2EDuration="2.064545553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 03:26:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:26:02.063801576 +0000 UTC m=+2108.376825435" watchObservedRunningTime="2026-09-14 03:26:02.064545553 +0000 UTC m=+2108.377569371"
Sep 14 03:26:03.173200 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:03.173170    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kxpfw_6a144e24-363d-48b1-a4a0-b27d07c109ba/dns/0.log"
Sep 14 03:26:03.197309 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:03.197282    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kxpfw_6a144e24-363d-48b1-a4a0-b27d07c109ba/kube-rbac-proxy/0.log"
Sep 14 03:26:03.362405 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:03.362385    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vfx28_60ad1800-10ba-4d33-a609-cb0e66dfdac5/dns-node-resolver/0.log"
Sep 14 03:26:04.040476 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:04.040443    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-84ls2_a851c347-abe6-4dce-8b64-5b40a3a812a8/node-ca/0.log"
Sep 14 03:26:05.090851 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:05.090816    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87_72ad94c5-039e-4ea7-a293-5aa10ed606b8/istio-proxy/0.log"
Sep 14 03:26:05.721858 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:05.721830    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hdvvq_ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795/istio-proxy/0.log"
Sep 14 03:26:05.729551 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:05.729504    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hdvvq_ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795/istio-proxy/1.log"
Sep 14 03:26:06.655940 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:06.655903    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7dxw9_d85ac4b3-b0a0-42d7-a99d-8170602d4170/serve-healthcheck-canary/0.log"
Sep 14 03:26:07.207382 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:07.207345    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-6gxb8_ae97ab3e-852d-4891-84bd-579e977e7aee/insights-operator/0.log"
Sep 14 03:26:07.208462 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:07.208444    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-6gxb8_ae97ab3e-852d-4891-84bd-579e977e7aee/insights-operator/1.log"
Sep 14 03:26:07.452733 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:07.452709    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p95kw_fedea9f9-1caf-464e-94e1-31ca4f616882/kube-rbac-proxy/0.log"
Sep 14 03:26:07.481982 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:07.481968    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p95kw_fedea9f9-1caf-464e-94e1-31ca4f616882/exporter/0.log"
Sep 14 03:26:07.517273 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:07.517253    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p95kw_fedea9f9-1caf-464e-94e1-31ca4f616882/extractor/0.log"
Sep 14 03:26:08.052823 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:08.052798    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-tmwf7"
Sep 14 03:26:09.973860 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:09.973804    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-mnsrl_d37c2e20-7fd9-4719-9b81-1f35e2f9e62c/manager/0.log"
Sep 14 03:26:10.650089 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:10.650018    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f647699c6-tjkbn_0ef0c507-14d9-4be4-9bc5-9d680635b0d9/manager/0.log"
Sep 14 03:26:10.667959 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:10.667940    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f647699c6-tjkbn_0ef0c507-14d9-4be4-9bc5-9d680635b0d9/copy-manifests/0.log"
Sep 14 03:26:11.058204 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:11.058171    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87_72ad94c5-039e-4ea7-a293-5aa10ed606b8/istio-proxy/0.log"
Sep 14 03:26:11.658707 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:11.658670    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hdvvq_ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795/istio-proxy/0.log"
Sep 14 03:26:11.666485 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:11.666459    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hdvvq_ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795/istio-proxy/1.log"
Sep 14 03:26:11.954850 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:11.954816    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5647bd797c-wcxbp_2d7b7c27-24d7-45ea-94b8-ff3e2bb37e5d/manager/0.log"
Sep 14 03:26:13.190777 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:13.190751    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87_72ad94c5-039e-4ea7-a293-5aa10ed606b8/istio-proxy/0.log"
Sep 14 03:26:13.826758 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:13.826726    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hdvvq_ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795/istio-proxy/0.log"
Sep 14 03:26:13.833664 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:13.833643    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hdvvq_ea0da5e4-6cc3-4bf1-99bb-a9b46ad9f795/istio-proxy/1.log"
Sep 14 03:26:17.489160 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:17.489124    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-m9bnr_c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b/kube-storage-version-migrator-operator/1.log"
Sep 14 03:26:17.489858 ip-10-0-133-188 kubenswrapper[2574]: I0914 03:26:17.489838    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-m9bnr_c2e2f2db-8ee5-45e5-b800-5c7b26aa3b7b/kube-storage-version-migrator-operator/0.log"