Sep 09 12:19:39.189194 ip-10-0-139-173 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 12:19:39.622425 ip-10-0-139-173 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:19:39.622861 ip-10-0-139-173 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 12:19:39.622861 ip-10-0-139-173 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:19:39.622861 ip-10-0-139-173 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 12:19:39.622861 ip-10-0-139-173 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:19:39.624836 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.624764    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 12:19:39.630666 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630646    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:19:39.630666 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630663    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:19:39.630666 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630667    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:19:39.630666 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630670    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630673    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630677    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630680    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630682    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630686    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630688    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630691    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630693    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630696    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630698    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630701    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630703    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630706    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630708    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630711    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630714    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630716    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630719    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:19:39.630808 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630722    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630725    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630731    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630733    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630736    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630738    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630741    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630743    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630746    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630752    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630757    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630760    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630762    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630765    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630769    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630772    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630775    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630778    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630781    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630783    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:19:39.631321 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630786    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630789    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630791    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630794    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630796    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630799    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630801    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630803    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630806    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630808    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630811    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630813    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630816    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630819    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630821    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630824    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630826    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630829    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630832    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630834    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:19:39.631806 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630837    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630840    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630844    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630846    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630849    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630853    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630855    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630860    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630863    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630866    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630868    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630871    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630873    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630876    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630878    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630881    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630883    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630886    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630888    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630890    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:19:39.632307 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630893    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630896    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630899    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630902    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630904    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630906    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630909    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630911    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630914    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630916    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630918    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630922    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630940    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630943    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630946    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630948    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.630951    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631657    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631664    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631668    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:19:39.632776 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631671    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631673    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631676    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631679    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631681    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631684    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631686    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631689    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631691    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631694    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631696    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631699    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631701    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631704    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631708    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631710    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631713    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631715    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631718    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631720    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:19:39.633270 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631723    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631726    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631728    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631731    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631733    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631736    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631739    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631741    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631744    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631747    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631749    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631752    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631755    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631757    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631759    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631762    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631765    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631767    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631770    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631772    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:19:39.633766 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631775    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631777    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631779    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631784    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631788    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631790    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631793    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631796    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631799    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631801    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631804    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631806    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631809    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631811    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631814    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631817    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631819    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631822    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631825    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:19:39.634263 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631827    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631830    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631832    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631834    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631837    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631839    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631842    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631844    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631846    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631849    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631851    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631854    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631856    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631860    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631864    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631868    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631871    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631873    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631876    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631880    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:19:39.634744 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631883    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631886    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631889    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631891    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631894    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631896    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631900    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631904    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631907    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631910    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631913    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631916    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631918    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631921    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631924    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631942    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.631944    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632315    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632320    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632323    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:19:39.635284 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632325    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632328    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632330    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632333    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632335    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632338    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632341    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632343    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632346    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632349    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632354    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632356    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632359    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632362    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632365    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632367    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632369    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632372    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632374    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:19:39.635780 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632377    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632380    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632383    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632385    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632388    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632390    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632392    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632395    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632397    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632400    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632402    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632405    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632408    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632410    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632413    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632415    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632418    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632420    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632425    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:19:39.636281 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632428    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632431    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632435    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632437    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632442    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632445    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632447    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632450    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632454    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632456    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632459    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632461    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632464    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632466    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632469    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632471    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632474    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632476    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632478    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632481    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:19:39.636759 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632483    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632485    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632488    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632490    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632493    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632495    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632497    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632501    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632503    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632506    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632508    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632510    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632513    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632515    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632518    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632520    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632524    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632527    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632529    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632532    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:19:39.637265 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632534    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632538    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632541    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632543    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632545    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632548    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632550    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632553    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632555    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632557    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632560    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632562    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632565    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632567    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632569    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632573    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632576    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.632579    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632654    2582 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632687    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632693    2582 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 12:19:39.637876 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632697    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632701    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632704    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632708    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632712    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632715    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632718    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632723    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632726    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632729    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632732    2582 flags.go:64] FLAG: --cgroup-root=""
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632734    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632737    2582 flags.go:64] FLAG: --client-ca-file=""
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632741    2582 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632744    2582 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632748    2582 flags.go:64] FLAG: --cluster-domain=""
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632752    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632754    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632757    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632761    2582 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632764    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632767    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632769    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 12:19:39.638407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632773    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632776    2582 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632779    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632782    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632785    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632788    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632791    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632794    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632797    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632799    2582 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632802    2582 flags.go:64] FLAG: --enable-server="true"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632805    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632809    2582 flags.go:64] FLAG: --event-burst="100"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632811    2582 flags.go:64] FLAG: --event-qps="50"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632814    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632817    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632820    2582 flags.go:64] FLAG: --eviction-hard=""
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632824    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632826    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632829    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632832    2582 flags.go:64] FLAG: --eviction-soft=""
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632834    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632837    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632840    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632843    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 12:19:39.638977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632845    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632848    2582 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632851    2582 flags.go:64] FLAG: --feature-gates=""
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632854    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632857    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632860    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632862    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632865    2582 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632868    2582 flags.go:64] FLAG: --help="false"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632871    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-139-173.ec2.internal"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632874    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632877    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632879    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632882    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632885    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632888    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632891    2582 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632893    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632896    2582 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632898    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632901    2582 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632905    2582 flags.go:64] FLAG: --kube-reserved=""
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632908    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632911    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 12:19:39.639596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632914    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632916    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632919    2582 flags.go:64] FLAG: --lock-file=""
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632922    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632939    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632947    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632954    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632958    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632961    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632964    2582 flags.go:64] FLAG: --logging-format="text"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632966    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632969    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632972    2582 flags.go:64] FLAG: --manifest-url=""
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632975    2582 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632979    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632982    2582 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632986    2582 flags.go:64] FLAG: --max-pods="110"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632989    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632992    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632994    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.632997    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633000    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633002    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633005    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633012    2582 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 12:19:39.640232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633015    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633017    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633020    2582 flags.go:64] FLAG: --pod-cidr=""
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633023    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633027    2582 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633032    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633035    2582 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633037    2582 flags.go:64] FLAG: --port="10250"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633040    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633043    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0eb84606e1657cc55"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633046    2582 flags.go:64] FLAG: --qos-reserved=""
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633049    2582 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633052    2582 flags.go:64] FLAG: --register-node="true"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633054    2582 flags.go:64] FLAG: --register-with-taints=""
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633058    2582 flags.go:64] FLAG: --registry-burst="10"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633062    2582 flags.go:64] FLAG: --registry-qps="5"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633064    2582 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633067    2582 flags.go:64] FLAG: --reserved-memory=""
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633070    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633073    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633076    2582 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633079    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633081    2582 flags.go:64] FLAG: --runonce="false"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633084    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633087    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 12:19:39.641239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633090    2582 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633093    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633095    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633098    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633101    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633103    2582 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633106    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633109    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633111    2582 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633114    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633117    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633120    2582 flags.go:64] FLAG: --system-cgroups=""
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633124    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633129    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633132    2582 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633135    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633139    2582 flags.go:64] FLAG: --tls-min-version=""
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633141    2582 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633144    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633147    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633149    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633152    2582 flags.go:64] FLAG: --v="2"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633156    2582 flags.go:64] FLAG: --version="false"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633164    2582 flags.go:64] FLAG: --vmodule=""
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633167    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 12:19:39.642372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.633170    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633280    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633285    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633288    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633291    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633293    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633296    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633299    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633301    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633304    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633306    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633308    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633311    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633314    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633318    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633321    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633323    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633326    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633329    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:19:39.643131 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633334    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633339    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633342    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633345    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633347    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633350    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633352    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633354    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633357    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633360    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633362    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633365    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633368    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633371    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633373    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633376    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633378    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633380    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633383    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633385    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:19:39.643698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633388    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633390    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633393    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633395    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633397    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633400    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633402    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633405    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633407    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633410    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633412    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633415    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633418    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633422    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633424    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633427    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633429    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633432    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633434    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633437    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:19:39.644577 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633439    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633442    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633444    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633446    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633449    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633452    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633455    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633459    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633462    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633465    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633468    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633470    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633473    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633475    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633478    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633480    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633483    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633485    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633488    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:19:39.645234 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633490    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633493    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633496    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633498    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633500    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633504    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633507    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633510    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633513    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633515    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633518    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633520    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633522    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633525    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633527    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633530    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633532    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633535    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633537    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:19:39.645775 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633543    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:19:39.646600 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633546    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:19:39.646600 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.633549    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:19:39.646600 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.634300    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:19:39.646600 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.642590    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 12:19:39.646600 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.642609    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 12:19:39.646600 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.643493    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 12:19:39.646600 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.643506    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 12:19:39.646600 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643590    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:19:39.646600 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643599    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:19:39.646600 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643605    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:19:39.646600 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643610    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:19:39.646600 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643614    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:19:39.646600 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643619    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:19:39.646600 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643624    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643628    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643633    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643638    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643642    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643647    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643651    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643655    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643659    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643663    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643667    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643674    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643680    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643685    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643689    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643694    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643698    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643702    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643708    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643712    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643717    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:19:39.647225 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643721    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643726    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643731    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643736    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643740    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643744    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643748    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643752    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643756    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643761    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643765    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643769    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643773    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643776    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643780    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643786    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643790    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643795    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643801    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:19:39.647735 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643807    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643812    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643816    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643820    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643824    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643827    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643832    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643836    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643840    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643843    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643847    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643852    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643856    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643861    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643865    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643870    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643874    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643878    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643882    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:19:39.648357 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643886    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643890    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643894    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643898    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643902    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643906    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643910    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643914    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643918    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643922    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643942    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643946    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643950    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643957    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643962    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643966    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643971    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643975    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643979    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:19:39.648821 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643984    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:19:39.649371 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643988    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:19:39.649371 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643992    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:19:39.649371 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.643996    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:19:39.649371 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644000    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:19:39.649371 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644005    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:19:39.649371 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644011    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:19:39.649371 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644015    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:19:39.649371 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644019    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:19:39.649371 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644024    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:19:39.649371 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644028    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:19:39.649371 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644032    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:19:39.649371 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644036    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:19:39.649371 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644041    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:19:39.649371 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644045    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:19:39.649713 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.644053    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:19:39.649713 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644231    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:19:39.649713 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644238    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:19:39.649713 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644242    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:19:39.649713 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644247    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:19:39.649713 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644252    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:19:39.649713 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644257    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:19:39.649713 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644261    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:19:39.649713 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644265    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:19:39.649713 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644269    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:19:39.649713 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644272    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:19:39.649713 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644276    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:19:39.649713 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644280    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:19:39.649713 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644284    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:19:39.649713 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644288    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644293    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644297    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644301    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644305    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644309    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644313    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644317    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644323    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644329    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644335    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644340    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644344    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644348    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644352    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644356    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644360    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644364    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644368    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:19:39.650173 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644372    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644376    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644380    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644385    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644390    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644394    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644398    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644402    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644406    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644410    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644414    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644418    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644422    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644426    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644430    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644435    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644439    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644443    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644447    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644451    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:19:39.650632 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644455    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644459    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644463    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644467    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644472    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644476    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644481    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644485    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644489    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644493    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644496    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644500    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644504    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644508    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644512    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644516    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644520    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644523    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644527    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644532    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:19:39.651248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644536    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644540    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644544    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644548    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644552    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644556    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644560    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644564    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644567    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644573    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644577    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644581    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644584    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644588    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644592    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644595    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644599    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644604    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644611    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:19:39.651757 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644616    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:19:39.652559 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644621    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:19:39.652559 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644626    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:19:39.652559 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644630    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:19:39.652559 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644634    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:19:39.652559 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644639    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:19:39.652559 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644643    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:19:39.652559 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:39.644647    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:19:39.652559 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.644655    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:19:39.652559 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.644803    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 12:19:39.652559 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.647979    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 12:19:39.652559 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.649122    2582 server.go:1019] "Starting client certificate rotation"
Sep 09 12:19:39.652559 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.649234    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 12:19:39.652559 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.649273    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 12:19:39.653130 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.652365    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 12:19:39.653759 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.653737    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 12:19:39.662309 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.662293    2582 log.go:25] "Validated CRI v1 runtime API"
Sep 09 12:19:39.665693 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.665679    2582 log.go:25] "Validated CRI v1 image API"
Sep 09 12:19:39.666549 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.666531    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 12:19:39.670685 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.670664    2582 fs.go:135] Filesystem UUIDs: map[6acdd26a-5ae4-4b14-9abf-5074b8fc5dc6:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7ea468cd-8b1c-41bf-b66f-abc72523a4f4:/dev/nvme0n1p3]
Sep 09 12:19:39.670762 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.670684    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 12:19:39.676140 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.676030    2582 manager.go:216] Machine: {Timestamp:2026-09-09 12:19:39.674607221 +0000 UTC m=+0.376093263 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099952 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec264e1477da22d6e45f9969ac27e995 SystemUUID:ec264e14-77da-22d6-e45f-9969ac27e995 BootID:ef26c360-af22-425b-8610-e1dcd7355c6e Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a6:42:2a:8d:9f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a6:42:2a:8d:9f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c6:7e:07:d9:44:95 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 12:19:39.676140 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.676129    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 12:19:39.676299 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.676258    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 12:19:39.679852 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.679826    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 12:19:39.680011 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.679856    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-173.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 12:19:39.681567 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.681557    2582 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 12:19:39.681609 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.681570    2582 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 12:19:39.681609 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.681578    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 12:19:39.682419 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.682408    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 12:19:39.684315 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.684305    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 12:19:39.688994 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.688979    2582 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 12:19:39.689136 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.689126    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 12:19:39.689167 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.689158    2582 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 12:19:39.689194 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.689168    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 12:19:39.689194 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.689181    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 12:19:39.689194 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.689190    2582 kubelet.go:398] "Adding apiserver pod source"
Sep 09 12:19:39.689280 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.689199    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 12:19:39.689525 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.689511    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 12:19:39.689596 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.689539    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 12:19:39.691008 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.690991    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 12:19:39.693123 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.693104    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 12:19:39.696123 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.696107    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 12:19:39.696204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.696137    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 12:19:39.696299 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.696287    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 12:19:39.696350 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.696311    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 12:19:39.696350 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.696320    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 12:19:39.696350 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.696325    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 12:19:39.696350 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.696334    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 12:19:39.696350 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.696339    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 12:19:39.697037 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.697026    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 12:19:39.697037 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.697037    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 12:19:39.697120 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.697045    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 12:19:39.697120 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.697050    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 12:19:39.697120 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.697059    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 12:19:39.697120 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.697067    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 12:19:39.697961 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.697952    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 12:19:39.697961 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.697962    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 12:19:39.699897 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.699867    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 12:19:39.699897 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.699868    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-173.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 12:19:39.702016 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.701995    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-173.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 12:19:39.703271 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.703259    2582 server.go:1268] "Started kubelet"
Sep 09 12:19:39.703358 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.703330    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 12:19:39.703409 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.703340    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 12:19:39.703455 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.703409    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 12:19:39.704053 ip-10-0-139-173 systemd[1]: Started Kubernetes Kubelet.
Sep 09 12:19:39.704389 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.704373    2582 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 12:19:39.704906 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.704887    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 12:19:39.710730 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.710709    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 12:19:39.711337 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.711319    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 12:19:39.711337 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.711334    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 12:19:39.711472 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.711394    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
Sep 09 12:19:39.712078 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.712062    2582 factory.go:55] Registering systemd factory
Sep 09 12:19:39.712078 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.712081    2582 factory.go:223] Registration of the systemd container factory successfully
Sep 09 12:19:39.712215 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.712111    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 12:19:39.712215 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.712114    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 12:19:39.712215 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.712133    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 12:19:39.712315 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.712237    2582 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 12:19:39.712315 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.712245    2582 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 12:19:39.712315 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.712306    2582 factory.go:153] Registering CRI-O factory
Sep 09 12:19:39.712315 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.712315    2582 factory.go:223] Registration of the crio container factory successfully
Sep 09 12:19:39.712436 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.712350    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 12:19:39.712436 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.712363    2582 factory.go:103] Registering Raw factory
Sep 09 12:19:39.712436 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.712373    2582 manager.go:1196] Started watching for new ooms in manager
Sep 09 12:19:39.712685 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.712674    2582 manager.go:349] Starting recovery of all containers
Sep 09 12:19:39.715180 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.715140    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-173.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 09 12:19:39.715356 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.715329    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 09 12:19:39.716489 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.715292    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-173.ec2.internal.18d3a6595ae13ddc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-173.ec2.internal,UID:ip-10-0-139-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-173.ec2.internal,},FirstTimestamp:2026-09-09 12:19:39.703229916 +0000 UTC m=+0.404715957,LastTimestamp:2026-09-09 12:19:39.703229916 +0000 UTC m=+0.404715957,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-173.ec2.internal,}"
Sep 09 12:19:39.724823 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.724805    2582 manager.go:354] Recovery completed
Sep 09 12:19:39.729026 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.729014    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:19:39.731242 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.731208    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:19:39.731295 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.731260    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:19:39.731295 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.731276    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:19:39.731765 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.731752    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 12:19:39.731813 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.731767    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 12:19:39.731813 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.731787    2582 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 12:19:39.731813 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.731794    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:19:39.732595 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.732581    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:19:39.732637 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.732607    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:19:39.732637 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.732618    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:19:39.733259 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.733203    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-173.ec2.internal.18d3a6595c8cac72  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-173.ec2.internal,UID:ip-10-0-139-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-173.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-173.ec2.internal,},FirstTimestamp:2026-09-09 12:19:39.731242098 +0000 UTC m=+0.432728141,LastTimestamp:2026-09-09 12:19:39.731242098 +0000 UTC m=+0.432728141,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-173.ec2.internal,}"
Sep 09 12:19:39.733981 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.733970    2582 policy_none.go:49] "None policy: Start"
Sep 09 12:19:39.734027 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.733985    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 12:19:39.734027 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.733995    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 12:19:39.735959 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.735948    2582 policy_none.go:47] "Start"
Sep 09 12:19:39.742504 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.742443    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-173.ec2.internal.18d3a6595c8d10b8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-173.ec2.internal,UID:ip-10-0-139-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-173.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-173.ec2.internal,},FirstTimestamp:2026-09-09 12:19:39.731267768 +0000 UTC m=+0.432753810,LastTimestamp:2026-09-09 12:19:39.731267768 +0000 UTC m=+0.432753810,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-173.ec2.internal,}"
Sep 09 12:19:39.745760 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.745694    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-173.ec2.internal.18d3a6595c8d484e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-173.ec2.internal,UID:ip-10-0-139-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-173.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-173.ec2.internal,},FirstTimestamp:2026-09-09 12:19:39.731281998 +0000 UTC m=+0.432768044,LastTimestamp:2026-09-09 12:19:39.731281998 +0000 UTC m=+0.432768044,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-173.ec2.internal,}"
Sep 09 12:19:39.752725 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.752667    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-139-173.ec2.internal.18d3a6595c8cac72\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-173.ec2.internal.18d3a6595c8cac72  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-173.ec2.internal,UID:ip-10-0-139-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-173.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-173.ec2.internal,},FirstTimestamp:2026-09-09 12:19:39.731242098 +0000 UTC m=+0.432728141,LastTimestamp:2026-09-09 12:19:39.732595824 +0000 UTC m=+0.434081864,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-173.ec2.internal,}"
Sep 09 12:19:39.754288 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.754268    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bdbhb"
Sep 09 12:19:39.761456 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.761320    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-139-173.ec2.internal.18d3a6595c8d10b8\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-173.ec2.internal.18d3a6595c8d10b8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-173.ec2.internal,UID:ip-10-0-139-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-173.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-173.ec2.internal,},FirstTimestamp:2026-09-09 12:19:39.731267768 +0000 UTC m=+0.432753810,LastTimestamp:2026-09-09 12:19:39.732612707 +0000 UTC m=+0.434098748,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-173.ec2.internal,}"
Sep 09 12:19:39.764258 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.764243    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bdbhb"
Sep 09 12:19:39.777460 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.777446    2582 manager.go:337] "Starting Device Plugin manager"
Sep 09 12:19:39.794363 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.777486    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 12:19:39.794363 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.777499    2582 server.go:87] "Starting device plugin registration server"
Sep 09 12:19:39.794363 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.777701    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 12:19:39.794363 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.777711    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 12:19:39.794363 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.777806    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 12:19:39.794363 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.777882    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 12:19:39.794363 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.777891    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 12:19:39.794363 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.778454    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 12:19:39.794363 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.778492    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-173.ec2.internal\" not found"
Sep 09 12:19:39.811896 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.811877    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 12:19:39.813105 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.813085    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 12:19:39.813191 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.813128    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 12:19:39.813191 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.813146    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 12:19:39.813191 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.813178    2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 09 12:19:39.815203 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.815189    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:19:39.878119 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.878078    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:19:39.878851 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.878832    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:19:39.878941 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.878858    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:19:39.878941 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.878868    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:19:39.878941 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.878889    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-173.ec2.internal"
Sep 09 12:19:39.886859 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.886844    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-173.ec2.internal"
Sep 09 12:19:39.886921 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.886864    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-173.ec2.internal\": node \"ip-10-0-139-173.ec2.internal\" not found"
Sep 09 12:19:39.901049 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.901025    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
Sep 09 12:19:39.913391 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.913371    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal"]
Sep 09 12:19:39.913469 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.913450    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:19:39.914754 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.914741    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:19:39.914820 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.914766    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:19:39.914820 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.914777    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:19:39.917054 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.917041    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:19:39.917185 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.917171    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
Sep 09 12:19:39.917227 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.917206    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:19:39.917648 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.917633    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:19:39.917730 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.917654    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:19:39.917730 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.917665    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:19:39.917730 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.917683    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:19:39.917730 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.917692    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:19:39.917730 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.917669    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:19:39.919858 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.919845    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal"
Sep 09 12:19:39.919914 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.919868    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:19:39.920484 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.920468    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:19:39.920573 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.920489    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:19:39.920573 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:39.920502    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:19:39.948425 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.948401    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-173.ec2.internal\" not found" node="ip-10-0-139-173.ec2.internal"
Sep 09 12:19:39.952629 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:39.952614    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-173.ec2.internal\" not found" node="ip-10-0-139-173.ec2.internal"
Sep 09 12:19:40.001367 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:40.001350    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
Sep 09 12:19:40.101684 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:40.101662    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
Sep 09 12:19:40.113021 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.113001    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0512c5589b44ea46a23646a68804fa89-config\") pod \"kube-apiserver-proxy-ip-10-0-139-173.ec2.internal\" (UID: \"0512c5589b44ea46a23646a68804fa89\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal"
Sep 09 12:19:40.113125 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.113026    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6c711764bbcbf3000f1d5e41efdec0f5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal\" (UID: \"6c711764bbcbf3000f1d5e41efdec0f5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
Sep 09 12:19:40.113125 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.113046    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c711764bbcbf3000f1d5e41efdec0f5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal\" (UID: \"6c711764bbcbf3000f1d5e41efdec0f5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
Sep 09 12:19:40.202396 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:40.202342    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
Sep 09 12:19:40.213639 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.213618    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6c711764bbcbf3000f1d5e41efdec0f5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal\" (UID: \"6c711764bbcbf3000f1d5e41efdec0f5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
Sep 09 12:19:40.213695 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.213644    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c711764bbcbf3000f1d5e41efdec0f5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal\" (UID: \"6c711764bbcbf3000f1d5e41efdec0f5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
Sep 09 12:19:40.213695 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.213661    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0512c5589b44ea46a23646a68804fa89-config\") pod \"kube-apiserver-proxy-ip-10-0-139-173.ec2.internal\" (UID: \"0512c5589b44ea46a23646a68804fa89\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal"
Sep 09 12:19:40.213695 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.213689    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0512c5589b44ea46a23646a68804fa89-config\") pod \"kube-apiserver-proxy-ip-10-0-139-173.ec2.internal\" (UID: \"0512c5589b44ea46a23646a68804fa89\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal"
Sep 09 12:19:40.213790 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.213723    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6c711764bbcbf3000f1d5e41efdec0f5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal\" (UID: \"6c711764bbcbf3000f1d5e41efdec0f5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
Sep 09 12:19:40.213790 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.213728    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c711764bbcbf3000f1d5e41efdec0f5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal\" (UID: \"6c711764bbcbf3000f1d5e41efdec0f5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
Sep 09 12:19:40.250782 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.250756    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
Sep 09 12:19:40.256395 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.256363    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal"
Sep 09 12:19:40.303417 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:40.303386    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
Sep 09 12:19:40.403897 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:40.403867    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
Sep 09 12:19:40.504455 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:40.504401    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
Sep 09 12:19:40.604990 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:40.604964    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
Sep 09 12:19:40.648487 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.648465    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 12:19:40.648839 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.648602    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 12:19:40.705043 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:40.705022    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
Sep 09 12:19:40.767053 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.767004    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 12:14:40 +0000 UTC" deadline="2028-06-01 20:44:03.66360831 +0000 UTC"
Sep 09 12:19:40.767148 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.767056    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15152h24m22.896557366s"
Sep 09 12:19:40.785734 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.785708    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:19:40.805298 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:40.805278    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
Sep 09 12:19:40.854734 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.854716    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:19:40.855647 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.855632    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:19:40.855712 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.855659    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:19:40.855712 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.855669    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:19:40.880377 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:40.880352    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0512c5589b44ea46a23646a68804fa89.slice/crio-cc3cb1d626e3fe32865ba450f5a3383457517ce57c821450c4e3ddd85ec12a24 WatchSource:0}: Error finding container cc3cb1d626e3fe32865ba450f5a3383457517ce57c821450c4e3ddd85ec12a24: Status 404 returned error can't find the container with id cc3cb1d626e3fe32865ba450f5a3383457517ce57c821450c4e3ddd85ec12a24
Sep 09 12:19:40.884291 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.884277    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:19:40.905631 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:40.905612    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
Sep 09 12:19:40.958944 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.958910    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:19:40.993552 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:40.993533    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:19:41.011980 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:41.011963    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
Sep 09 12:19:41.022872 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:41.022837    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 12:19:41.024010 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:41.023998    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal"
Sep 09 12:19:41.032268 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:41.032251    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 12:19:42.563225 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.563155    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:19:42.694952 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.694920    2582 apiserver.go:52] "Watching apiserver"
Sep 09 12:19:42.700351 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.700329    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 12:19:42.700757 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.700729    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-4kv6h","openshift-network-operator/iptables-alerter-pd4tx","kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal","openshift-dns/node-resolver-8zf7j","openshift-image-registry/node-ca-bjqck","openshift-multus/multus-dpwzk","openshift-network-diagnostics/network-check-target-b9h5d","openshift-ovn-kubernetes/ovnkube-node-x9vqw","kube-system/global-pull-secret-syncer-lxbw5","kube-system/konnectivity-agent-tqtbz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb","openshift-cluster-node-tuning-operator/tuned-vjlwf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal","openshift-multus/multus-additional-cni-plugins-8tjr5"]
Sep 09 12:19:42.703212 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.703184    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:42.703315 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:42.703270    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:19:42.705292 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.705269    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.707017 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.706964    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 12:19:42.707554 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.707330    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-d9plq\""
Sep 09 12:19:42.707554 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.707341    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 12:19:42.707554 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.707372    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 12:19:42.707711 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.707569    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.709468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.709442    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 12:19:42.709738 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.709693    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8zf7j"
Sep 09 12:19:42.713049 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.710312    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 12:19:42.713049 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.710446    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 12:19:42.713049 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.711093    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 12:19:42.713049 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.711106    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bdhdk\""
Sep 09 12:19:42.713049 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.711226    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 12:19:42.713049 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.712160    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tmdcd\""
Sep 09 12:19:42.713049 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.712599    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 12:19:42.713049 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.713041    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 12:19:42.714298 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.714277    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bjqck"
Sep 09 12:19:42.715943 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.715917    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-q2v6p\""
Sep 09 12:19:42.716118 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.716076    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 12:19:42.716246 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.716155    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 12:19:42.716246 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.716190    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 12:19:42.716460 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.716441    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.716640 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.716621    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 12:19:42.718142 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.718124    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-flrvr\""
Sep 09 12:19:42.718246 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.718143    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 12:19:42.719432 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.719414    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.721269 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.721248    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:19:42.721380 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.721353    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-f4f55\""
Sep 09 12:19:42.721448 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.721398    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 12:19:42.721996 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.721756    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:42.721996 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:42.721826    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:19:42.723741 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.723688    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-tmp\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.723741 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.723724    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18511945-5b00-4ea7-8c23-598b3f3de208-system-cni-dir\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.723923 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.723750    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18511945-5b00-4ea7-8c23-598b3f3de208-cni-binary-copy\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.723923 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.723774    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-cnibin\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.723923 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.723799    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-run-netns\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.723923 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.723825    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-etc-kubernetes\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.723923 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.723883    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-etc-selinux\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.723923 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.723899    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-sysconfig\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.723923 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.723915    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgszh\" (UniqueName: \"kubernetes.io/projected/ae8eed86-8b04-4c9a-ab62-4ea97a34849d-kube-api-access-sgszh\") pod \"node-ca-bjqck\" (UID: \"ae8eed86-8b04-4c9a-ab62-4ea97a34849d\") " pod="openshift-image-registry/node-ca-bjqck"
Sep 09 12:19:42.724253 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.723930    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-multus-socket-dir-parent\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.724253 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.723946    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-run-k8s-cni-cncf-io\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.724253 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.723969    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-run-multus-certs\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.724253 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.723992    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-sysctl-d\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.724253 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724064    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-multus-cni-dir\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.724253 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724111    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-device-dir\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.724253 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724175    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-sys\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.724253 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724202    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bcz9\" (UniqueName: \"kubernetes.io/projected/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-kube-api-access-2bcz9\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.724253 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724245    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/18511945-5b00-4ea7-8c23-598b3f3de208-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.724634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724280    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gxxv\" (UniqueName: \"kubernetes.io/projected/18511945-5b00-4ea7-8c23-598b3f3de208-kube-api-access-5gxxv\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.724634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724290    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pd4tx"
Sep 09 12:19:42.724634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724322    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2bxd\" (UniqueName: \"kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd\") pod \"network-check-target-b9h5d\" (UID: \"e2d11583-88d9-42e6-8de7-48df91ba8254\") " pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:42.724634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724357    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-socket-dir\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.724634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724382    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-sysctl-conf\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.724634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724405    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-tuned\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.724634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724438    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-system-cni-dir\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.724634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724462    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-var-lib-cni-multus\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.724634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724488    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-hostroot\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.724634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724513    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a2a8b0-4d6b-4171-b470-7fd9b57a16e8-tmp-dir\") pod \"node-resolver-8zf7j\" (UID: \"25a2a8b0-4d6b-4171-b470-7fd9b57a16e8\") " pod="openshift-dns/node-resolver-8zf7j"
Sep 09 12:19:42.724634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724537    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w2kj\" (UniqueName: \"kubernetes.io/projected/25a2a8b0-4d6b-4171-b470-7fd9b57a16e8-kube-api-access-6w2kj\") pod \"node-resolver-8zf7j\" (UID: \"25a2a8b0-4d6b-4171-b470-7fd9b57a16e8\") " pod="openshift-dns/node-resolver-8zf7j"
Sep 09 12:19:42.724634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724563    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18511945-5b00-4ea7-8c23-598b3f3de208-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.724634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724621    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/18511945-5b00-4ea7-8c23-598b3f3de208-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.725034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724650    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae8eed86-8b04-4c9a-ab62-4ea97a34849d-host\") pod \"node-ca-bjqck\" (UID: \"ae8eed86-8b04-4c9a-ab62-4ea97a34849d\") " pod="openshift-image-registry/node-ca-bjqck"
Sep 09 12:19:42.725034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724679    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-var-lib-kubelet\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.725034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724705    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-multus-daemon-config\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.725034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724732    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks57b\" (UniqueName: \"kubernetes.io/projected/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-kube-api-access-ks57b\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.725034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724760    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-modprobe-d\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.725034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724795    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-systemd\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.725034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724810    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-run\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.725034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724843    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-lib-modules\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.725034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724889    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18511945-5b00-4ea7-8c23-598b3f3de208-cnibin\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.725034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724914    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-cni-binary-copy\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.725034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724938    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-var-lib-cni-bin\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.725034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724965    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-multus-conf-dir\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.725034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.724993    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.725034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.725019    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-registration-dir\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.726328 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.725050    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-sys-fs\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.726328 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.725083    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5fnn\" (UniqueName: \"kubernetes.io/projected/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-kube-api-access-m5fnn\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.726328 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.725109    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-kubernetes\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.726328 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.725125    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-var-lib-kubelet\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.726328 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.725146    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ae8eed86-8b04-4c9a-ab62-4ea97a34849d-serviceca\") pod \"node-ca-bjqck\" (UID: \"ae8eed86-8b04-4c9a-ab62-4ea97a34849d\") " pod="openshift-image-registry/node-ca-bjqck"
Sep 09 12:19:42.726328 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.725168    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/25a2a8b0-4d6b-4171-b470-7fd9b57a16e8-hosts-file\") pod \"node-resolver-8zf7j\" (UID: \"25a2a8b0-4d6b-4171-b470-7fd9b57a16e8\") " pod="openshift-dns/node-resolver-8zf7j"
Sep 09 12:19:42.726328 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.725199    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-host\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.726328 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.726071    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 12:19:42.726328 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.726166    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 12:19:42.726328 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.726181    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:19:42.726328 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.726229    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 12:19:42.726756 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:42.726330    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c711764bbcbf3000f1d5e41efdec0f5.slice/crio-42fc07e4c11aab5b3d48f82175d90c52149d9dc12197c4bcfe0a6aeba8f82037 WatchSource:0}: Error finding container 42fc07e4c11aab5b3d48f82175d90c52149d9dc12197c4bcfe0a6aeba8f82037: Status 404 returned error can't find the container with id 42fc07e4c11aab5b3d48f82175d90c52149d9dc12197c4bcfe0a6aeba8f82037
Sep 09 12:19:42.726756 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.726471    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cxdms\""
Sep 09 12:19:42.727067 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.727023    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tqtbz"
Sep 09 12:19:42.729025 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.729006    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 12:19:42.729150 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.729033    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t5bnl\""
Sep 09 12:19:42.729150 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.729009    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 12:19:42.729308 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.729265    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:42.729368 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:42.729330    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:19:42.731743 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.731724    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.733670 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.733646    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 12:19:42.733811 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.733791    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 12:19:42.733949 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.733648    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 12:19:42.734059 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.734031    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ffcgm\""
Sep 09 12:19:42.734059 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.734041    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 12:19:42.734259 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.734154    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 12:19:42.734259 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.734196    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 12:19:42.767776 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.767758    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fcxff"
Sep 09 12:19:42.773512 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.773495    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fcxff"
Sep 09 12:19:42.819692 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.819644    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 12:19:42.823036 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.822993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal" event={"ID":"6c711764bbcbf3000f1d5e41efdec0f5","Type":"ContainerStarted","Data":"42fc07e4c11aab5b3d48f82175d90c52149d9dc12197c4bcfe0a6aeba8f82037"}
Sep 09 12:19:42.825791 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.825771    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-etc-selinux\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.825914 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.825822    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-sysconfig\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.825914 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.825870    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-sysctl-d\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.825914 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.825898    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-env-overrides\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.826073 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.825922    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-dbus\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:42.826073 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.825953    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-sys\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.826073 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.825977    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/18511945-5b00-4ea7-8c23-598b3f3de208-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.826073 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826003    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gxxv\" (UniqueName: \"kubernetes.io/projected/18511945-5b00-4ea7-8c23-598b3f3de208-kube-api-access-5gxxv\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.826073 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826026    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-socket-dir\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.826073 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826048    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-system-cni-dir\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.826355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826083    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-var-lib-cni-multus\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.826355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826108    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-hostroot\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.826355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826134    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-sysconfig\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.826355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826131    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-systemd-units\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.826355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826175    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-slash\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.826355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826198    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-run-netns\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.826355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826220    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-run-openvswitch\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.826355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826247    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a2a8b0-4d6b-4171-b470-7fd9b57a16e8-tmp-dir\") pod \"node-resolver-8zf7j\" (UID: \"25a2a8b0-4d6b-4171-b470-7fd9b57a16e8\") " pod="openshift-dns/node-resolver-8zf7j"
Sep 09 12:19:42.826355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826273    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w2kj\" (UniqueName: \"kubernetes.io/projected/25a2a8b0-4d6b-4171-b470-7fd9b57a16e8-kube-api-access-6w2kj\") pod \"node-resolver-8zf7j\" (UID: \"25a2a8b0-4d6b-4171-b470-7fd9b57a16e8\") " pod="openshift-dns/node-resolver-8zf7j"
Sep 09 12:19:42.826355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826296    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/18511945-5b00-4ea7-8c23-598b3f3de208-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.826355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826320    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-var-lib-kubelet\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.826355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826331    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-sysctl-d\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.826355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826348    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab073a4e-177f-48c0-9091-fb355d8bce3f-host-slash\") pod \"iptables-alerter-pd4tx\" (UID: \"ab073a4e-177f-48c0-9091-fb355d8bce3f\") " pod="openshift-network-operator/iptables-alerter-pd4tx"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826370    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-sys\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826371    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-log-socket\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826436    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-system-cni-dir\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826436    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-var-lib-kubelet\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826468    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-var-lib-cni-multus\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826486    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-hostroot\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826498    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-socket-dir\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826508    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-cni-bin\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826534    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5aabdb32-70b9-4bbf-a9ca-b15d76924dc7-agent-certs\") pod \"konnectivity-agent-tqtbz\" (UID: \"5aabdb32-70b9-4bbf-a9ca-b15d76924dc7\") " pod="kube-system/konnectivity-agent-tqtbz"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826654    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-etc-selinux\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826672    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a2a8b0-4d6b-4171-b470-7fd9b57a16e8-tmp-dir\") pod \"node-resolver-8zf7j\" (UID: \"25a2a8b0-4d6b-4171-b470-7fd9b57a16e8\") " pod="openshift-dns/node-resolver-8zf7j"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826672    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-modprobe-d\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826722    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-modprobe-d\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826730    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-systemd\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826746    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18511945-5b00-4ea7-8c23-598b3f3de208-cnibin\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826814    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal" event={"ID":"0512c5589b44ea46a23646a68804fa89","Type":"ContainerStarted","Data":"cc3cb1d626e3fe32865ba450f5a3383457517ce57c821450c4e3ddd85ec12a24"}
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826826    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-systemd\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.826940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826831    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18511945-5b00-4ea7-8c23-598b3f3de208-cnibin\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826888    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-run-systemd\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826919    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826943    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-registration-dir\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826968    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-sys-fs\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.826990    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-kubernetes\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827012    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ae8eed86-8b04-4c9a-ab62-4ea97a34849d-serviceca\") pod \"node-ca-bjqck\" (UID: \"ae8eed86-8b04-4c9a-ab62-4ea97a34849d\") " pod="openshift-image-registry/node-ca-bjqck"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827016    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827031    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-registration-dir\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827036    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs\") pod \"network-metrics-daemon-4kv6h\" (UID: \"5fa079e3-edaf-4746-93c9-fa89298bbb94\") " pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827067    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-kubelet\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827068    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-sys-fs\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827072    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-kubernetes\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827093    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-run-ovn\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827125    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-tmp\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827147    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18511945-5b00-4ea7-8c23-598b3f3de208-cni-binary-copy\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827170    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-cnibin\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.827590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827217    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-cnibin\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827268    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/18511945-5b00-4ea7-8c23-598b3f3de208-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827306    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-run-netns\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827334    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-etc-kubernetes\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827351    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827368    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgszh\" (UniqueName: \"kubernetes.io/projected/ae8eed86-8b04-4c9a-ab62-4ea97a34849d-kube-api-access-sgszh\") pod \"node-ca-bjqck\" (UID: \"ae8eed86-8b04-4c9a-ab62-4ea97a34849d\") " pod="openshift-image-registry/node-ca-bjqck"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827403    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-etc-kubernetes\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827412    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ae8eed86-8b04-4c9a-ab62-4ea97a34849d-serviceca\") pod \"node-ca-bjqck\" (UID: \"ae8eed86-8b04-4c9a-ab62-4ea97a34849d\") " pod="openshift-image-registry/node-ca-bjqck"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827427    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-run-netns\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827435    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-multus-socket-dir-parent\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827456    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-run-k8s-cni-cncf-io\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827436    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827471    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-run-multus-certs\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827492    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-run-ovn-kubernetes\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827518    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-ovn-node-metrics-cert\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827524    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-run-k8s-cni-cncf-io\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827542    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-multus-cni-dir\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827571    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs2vf\" (UniqueName: \"kubernetes.io/projected/5fa079e3-edaf-4746-93c9-fa89298bbb94-kube-api-access-cs2vf\") pod \"network-metrics-daemon-4kv6h\" (UID: \"5fa079e3-edaf-4746-93c9-fa89298bbb94\") " pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:42.828250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827572    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-multus-socket-dir-parent\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827588    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-run-multus-certs\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827591    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-node-log\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827604    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18511945-5b00-4ea7-8c23-598b3f3de208-cni-binary-copy\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827622    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-cni-netd\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827635    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-multus-cni-dir\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827654    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-device-dir\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827684    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-device-dir\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827686    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bcz9\" (UniqueName: \"kubernetes.io/projected/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-kube-api-access-2bcz9\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827710    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-etc-openvswitch\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827725    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-ovnkube-config\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827739    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x68gg\" (UniqueName: \"kubernetes.io/projected/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-kube-api-access-x68gg\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827759    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2bxd\" (UniqueName: \"kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd\") pod \"network-check-target-b9h5d\" (UID: \"e2d11583-88d9-42e6-8de7-48df91ba8254\") " pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827816    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-sysctl-conf\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827868    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-tuned\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827890    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcn67\" (UniqueName: \"kubernetes.io/projected/ab073a4e-177f-48c0-9091-fb355d8bce3f-kube-api-access-mcn67\") pod \"iptables-alerter-pd4tx\" (UID: \"ab073a4e-177f-48c0-9091-fb355d8bce3f\") " pod="openshift-network-operator/iptables-alerter-pd4tx"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827915    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-var-lib-openvswitch\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.829071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827937    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18511945-5b00-4ea7-8c23-598b3f3de208-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827942    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-sysctl-conf\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827955    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae8eed86-8b04-4c9a-ab62-4ea97a34849d-host\") pod \"node-ca-bjqck\" (UID: \"ae8eed86-8b04-4c9a-ab62-4ea97a34849d\") " pod="openshift-image-registry/node-ca-bjqck"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.827973    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-multus-daemon-config\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828051    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae8eed86-8b04-4c9a-ab62-4ea97a34849d-host\") pod \"node-ca-bjqck\" (UID: \"ae8eed86-8b04-4c9a-ab62-4ea97a34849d\") " pod="openshift-image-registry/node-ca-bjqck"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828094    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18511945-5b00-4ea7-8c23-598b3f3de208-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828138    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks57b\" (UniqueName: \"kubernetes.io/projected/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-kube-api-access-ks57b\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828201    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-ovnkube-script-lib\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828227    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-run\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828263    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-lib-modules\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828285    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-cni-binary-copy\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828306    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-var-lib-cni-bin\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828356    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-run\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828393    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-lib-modules\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828405    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-host-var-lib-cni-bin\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828440    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-multus-daemon-config\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828442    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-multus-conf-dir\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828471    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-multus-conf-dir\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.829577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828484    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5aabdb32-70b9-4bbf-a9ca-b15d76924dc7-konnectivity-ca\") pod \"konnectivity-agent-tqtbz\" (UID: \"5aabdb32-70b9-4bbf-a9ca-b15d76924dc7\") " pod="kube-system/konnectivity-agent-tqtbz"
Sep 09 12:19:42.830223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828507    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-kubelet-config\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:42.830223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828527    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:42.830223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828553    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5fnn\" (UniqueName: \"kubernetes.io/projected/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-kube-api-access-m5fnn\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.830223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828578    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-var-lib-kubelet\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.830223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828626    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ab073a4e-177f-48c0-9091-fb355d8bce3f-iptables-alerter-script\") pod \"iptables-alerter-pd4tx\" (UID: \"ab073a4e-177f-48c0-9091-fb355d8bce3f\") " pod="openshift-network-operator/iptables-alerter-pd4tx"
Sep 09 12:19:42.830223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828641    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-var-lib-kubelet\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.830223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828824    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/25a2a8b0-4d6b-4171-b470-7fd9b57a16e8-hosts-file\") pod \"node-resolver-8zf7j\" (UID: \"25a2a8b0-4d6b-4171-b470-7fd9b57a16e8\") " pod="openshift-dns/node-resolver-8zf7j"
Sep 09 12:19:42.830223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828880    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/25a2a8b0-4d6b-4171-b470-7fd9b57a16e8-hosts-file\") pod \"node-resolver-8zf7j\" (UID: \"25a2a8b0-4d6b-4171-b470-7fd9b57a16e8\") " pod="openshift-dns/node-resolver-8zf7j"
Sep 09 12:19:42.830223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828910    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-host\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.830223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828935    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18511945-5b00-4ea7-8c23-598b3f3de208-system-cni-dir\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.830223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828976    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-host\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.830223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.828917    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/18511945-5b00-4ea7-8c23-598b3f3de208-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.830223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.829007    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18511945-5b00-4ea7-8c23-598b3f3de208-system-cni-dir\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.830223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.829024    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-cni-binary-copy\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.830985 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.830966    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-tmp\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.831058 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.831019    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-etc-tuned\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.837062 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:42.836149    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:19:42.837062 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:42.836170    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:19:42.837062 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:42.836182    2582 projected.go:196] Error preparing data for projected volume kube-api-access-z2bxd for pod openshift-network-diagnostics/network-check-target-b9h5d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:42.837062 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:42.836244    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd podName:e2d11583-88d9-42e6-8de7-48df91ba8254 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:43.336217631 +0000 UTC m=+3.032782776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-z2bxd" (UniqueName: "kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd") pod "network-check-target-b9h5d" (UID: "e2d11583-88d9-42e6-8de7-48df91ba8254") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:42.837574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.837537    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w2kj\" (UniqueName: \"kubernetes.io/projected/25a2a8b0-4d6b-4171-b470-7fd9b57a16e8-kube-api-access-6w2kj\") pod \"node-resolver-8zf7j\" (UID: \"25a2a8b0-4d6b-4171-b470-7fd9b57a16e8\") " pod="openshift-dns/node-resolver-8zf7j"
Sep 09 12:19:42.838032 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.838008    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5fnn\" (UniqueName: \"kubernetes.io/projected/c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c-kube-api-access-m5fnn\") pod \"aws-ebs-csi-driver-node-2vbrb\" (UID: \"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:42.838898 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.838877    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgszh\" (UniqueName: \"kubernetes.io/projected/ae8eed86-8b04-4c9a-ab62-4ea97a34849d-kube-api-access-sgszh\") pod \"node-ca-bjqck\" (UID: \"ae8eed86-8b04-4c9a-ab62-4ea97a34849d\") " pod="openshift-image-registry/node-ca-bjqck"
Sep 09 12:19:42.840311 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.840289    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks57b\" (UniqueName: \"kubernetes.io/projected/3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c-kube-api-access-ks57b\") pod \"multus-dpwzk\" (UID: \"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c\") " pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:42.840926 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.840888    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gxxv\" (UniqueName: \"kubernetes.io/projected/18511945-5b00-4ea7-8c23-598b3f3de208-kube-api-access-5gxxv\") pod \"multus-additional-cni-plugins-8tjr5\" (UID: \"18511945-5b00-4ea7-8c23-598b3f3de208\") " pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:42.843016 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.842996    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bcz9\" (UniqueName: \"kubernetes.io/projected/141b4693-a55b-4c1d-89eb-f19b26a9e1f5-kube-api-access-2bcz9\") pod \"tuned-vjlwf\" (UID: \"141b4693-a55b-4c1d-89eb-f19b26a9e1f5\") " pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:42.929243 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929212    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5aabdb32-70b9-4bbf-a9ca-b15d76924dc7-konnectivity-ca\") pod \"konnectivity-agent-tqtbz\" (UID: \"5aabdb32-70b9-4bbf-a9ca-b15d76924dc7\") " pod="kube-system/konnectivity-agent-tqtbz"
Sep 09 12:19:42.929375 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929257    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-kubelet-config\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:42.929375 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929284    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:42.929375 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929327    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ab073a4e-177f-48c0-9091-fb355d8bce3f-iptables-alerter-script\") pod \"iptables-alerter-pd4tx\" (UID: \"ab073a4e-177f-48c0-9091-fb355d8bce3f\") " pod="openshift-network-operator/iptables-alerter-pd4tx"
Sep 09 12:19:42.929375 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929356    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-env-overrides\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.929574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929379    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-dbus\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:42.929574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929382    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-kubelet-config\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:42.929574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929428    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-systemd-units\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.929574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929448    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-slash\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.929574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929462    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-run-netns\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.929574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929476    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-run-openvswitch\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.929574 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:42.929482    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:42.929574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab073a4e-177f-48c0-9091-fb355d8bce3f-host-slash\") pod \"iptables-alerter-pd4tx\" (UID: \"ab073a4e-177f-48c0-9091-fb355d8bce3f\") " pod="openshift-network-operator/iptables-alerter-pd4tx"
Sep 09 12:19:42.929574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929506    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-log-socket\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.929574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929519    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-cni-bin\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.929574 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:42.929540    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret podName:cb9acdab-04f6-47b1-9bcd-d24246dcf52a nodeName:}" failed. No retries permitted until 2026-09-09 12:19:43.429521269 +0000 UTC m=+3.126086413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret") pod "global-pull-secret-syncer-lxbw5" (UID: "cb9acdab-04f6-47b1-9bcd-d24246dcf52a") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929571    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5aabdb32-70b9-4bbf-a9ca-b15d76924dc7-agent-certs\") pod \"konnectivity-agent-tqtbz\" (UID: \"5aabdb32-70b9-4bbf-a9ca-b15d76924dc7\") " pod="kube-system/konnectivity-agent-tqtbz"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929611    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-run-systemd\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929640    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs\") pod \"network-metrics-daemon-4kv6h\" (UID: \"5fa079e3-edaf-4746-93c9-fa89298bbb94\") " pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929686    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-kubelet\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929714    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-run-ovn\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929746    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929778    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-run-ovn-kubernetes\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929805    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-ovn-node-metrics-cert\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929833    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs2vf\" (UniqueName: \"kubernetes.io/projected/5fa079e3-edaf-4746-93c9-fa89298bbb94-kube-api-access-cs2vf\") pod \"network-metrics-daemon-4kv6h\" (UID: \"5fa079e3-edaf-4746-93c9-fa89298bbb94\") " pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929885    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-node-log\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929909    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-cni-netd\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929937    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-etc-openvswitch\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929953    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5aabdb32-70b9-4bbf-a9ca-b15d76924dc7-konnectivity-ca\") pod \"konnectivity-agent-tqtbz\" (UID: \"5aabdb32-70b9-4bbf-a9ca-b15d76924dc7\") " pod="kube-system/konnectivity-agent-tqtbz"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929964    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-ovnkube-config\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929999    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x68gg\" (UniqueName: \"kubernetes.io/projected/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-kube-api-access-x68gg\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930005    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-dbus\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:42.930081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930032    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-systemd-units\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930038    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcn67\" (UniqueName: \"kubernetes.io/projected/ab073a4e-177f-48c0-9091-fb355d8bce3f-kube-api-access-mcn67\") pod \"iptables-alerter-pd4tx\" (UID: \"ab073a4e-177f-48c0-9091-fb355d8bce3f\") " pod="openshift-network-operator/iptables-alerter-pd4tx"
Sep 09 12:19:42.930807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930053    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-slash\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930065    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-var-lib-openvswitch\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930074    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-run-netns\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930101    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-run-openvswitch\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930124    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-var-lib-openvswitch\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930126    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-ovnkube-script-lib\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930534    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-ovnkube-script-lib\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930578    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab073a4e-177f-48c0-9091-fb355d8bce3f-host-slash\") pod \"iptables-alerter-pd4tx\" (UID: \"ab073a4e-177f-48c0-9091-fb355d8bce3f\") " pod="openshift-network-operator/iptables-alerter-pd4tx"
Sep 09 12:19:42.930807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930602    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-log-socket\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930629    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.930807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930674    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ab073a4e-177f-48c0-9091-fb355d8bce3f-iptables-alerter-script\") pod \"iptables-alerter-pd4tx\" (UID: \"ab073a4e-177f-48c0-9091-fb355d8bce3f\") " pod="openshift-network-operator/iptables-alerter-pd4tx"
Sep 09 12:19:42.930807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930739    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-run-ovn-kubernetes\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.931420 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.930874    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-env-overrides\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.931420 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.929541    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-cni-bin\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.931420 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.931140    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-kubelet\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.931420 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.931149    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-etc-openvswitch\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.931420 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.931195    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-run-systemd\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.931420 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:42.931277    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:42.931420 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:42.931331    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs podName:5fa079e3-edaf-4746-93c9-fa89298bbb94 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:43.431315086 +0000 UTC m=+3.127880221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs") pod "network-metrics-daemon-4kv6h" (UID: "5fa079e3-edaf-4746-93c9-fa89298bbb94") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:42.931420 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.931365    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-run-ovn\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.931420 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.931383    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-node-log\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.931420 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.931396    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-host-cni-netd\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.932009 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.931989    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-ovnkube-config\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.933446 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.933423    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-ovn-node-metrics-cert\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:42.933559 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.933537    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5aabdb32-70b9-4bbf-a9ca-b15d76924dc7-agent-certs\") pod \"konnectivity-agent-tqtbz\" (UID: \"5aabdb32-70b9-4bbf-a9ca-b15d76924dc7\") " pod="kube-system/konnectivity-agent-tqtbz"
Sep 09 12:19:42.939374 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.939354    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs2vf\" (UniqueName: \"kubernetes.io/projected/5fa079e3-edaf-4746-93c9-fa89298bbb94-kube-api-access-cs2vf\") pod \"network-metrics-daemon-4kv6h\" (UID: \"5fa079e3-edaf-4746-93c9-fa89298bbb94\") " pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:42.941550 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.941524    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcn67\" (UniqueName: \"kubernetes.io/projected/ab073a4e-177f-48c0-9091-fb355d8bce3f-kube-api-access-mcn67\") pod \"iptables-alerter-pd4tx\" (UID: \"ab073a4e-177f-48c0-9091-fb355d8bce3f\") " pod="openshift-network-operator/iptables-alerter-pd4tx"
Sep 09 12:19:42.942149 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:42.942129    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x68gg\" (UniqueName: \"kubernetes.io/projected/8a0ff7f0-303a-45c2-b0f4-8081800ab66f-kube-api-access-x68gg\") pod \"ovnkube-node-x9vqw\" (UID: \"8a0ff7f0-303a-45c2-b0f4-8081800ab66f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:43.018092 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.018068    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb"
Sep 09 12:19:43.025623 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.025599    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8tjr5"
Sep 09 12:19:43.035396 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.035376    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8zf7j"
Sep 09 12:19:43.035587 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:43.035565    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18511945_5b00_4ea7_8c23_598b3f3de208.slice/crio-2ce6561755298fabb6219bb9ea891343d8511e63b69f0b7f2b92c8b601e1d2ba WatchSource:0}: Error finding container 2ce6561755298fabb6219bb9ea891343d8511e63b69f0b7f2b92c8b601e1d2ba: Status 404 returned error can't find the container with id 2ce6561755298fabb6219bb9ea891343d8511e63b69f0b7f2b92c8b601e1d2ba
Sep 09 12:19:43.045124 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.045100    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bjqck"
Sep 09 12:19:43.045802 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:43.045776    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a2a8b0_4d6b_4171_b470_7fd9b57a16e8.slice/crio-a97a08b5669fe50e77680f824585c443ae200140c330c9f725e065c119d6f87b WatchSource:0}: Error finding container a97a08b5669fe50e77680f824585c443ae200140c330c9f725e065c119d6f87b: Status 404 returned error can't find the container with id a97a08b5669fe50e77680f824585c443ae200140c330c9f725e065c119d6f87b
Sep 09 12:19:43.050481 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.050462    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dpwzk"
Sep 09 12:19:43.055590 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:43.055565    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae8eed86_8b04_4c9a_ab62_4ea97a34849d.slice/crio-29e05b0358ca3047244eeada5e8b0863cbd8c69b2392b1219c2a5d2cf4766df0 WatchSource:0}: Error finding container 29e05b0358ca3047244eeada5e8b0863cbd8c69b2392b1219c2a5d2cf4766df0: Status 404 returned error can't find the container with id 29e05b0358ca3047244eeada5e8b0863cbd8c69b2392b1219c2a5d2cf4766df0
Sep 09 12:19:43.057841 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.057484    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vjlwf"
Sep 09 12:19:43.059619 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:43.059598    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f6ae89a_46c0_4adc_ac1d_9cd0e286f25c.slice/crio-d747c932ae89ca698f759872b4c9f11979a58396517ae42abb3dd74b859c4c8d WatchSource:0}: Error finding container d747c932ae89ca698f759872b4c9f11979a58396517ae42abb3dd74b859c4c8d: Status 404 returned error can't find the container with id d747c932ae89ca698f759872b4c9f11979a58396517ae42abb3dd74b859c4c8d
Sep 09 12:19:43.064178 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.064155    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pd4tx"
Sep 09 12:19:43.069484 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.069462    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tqtbz"
Sep 09 12:19:43.071300 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:43.071246    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod141b4693_a55b_4c1d_89eb_f19b26a9e1f5.slice/crio-707f7fab9ea0ca6e101141af4c3d62f7fc892d4e539722734813bd9e6ca4ee48 WatchSource:0}: Error finding container 707f7fab9ea0ca6e101141af4c3d62f7fc892d4e539722734813bd9e6ca4ee48: Status 404 returned error can't find the container with id 707f7fab9ea0ca6e101141af4c3d62f7fc892d4e539722734813bd9e6ca4ee48
Sep 09 12:19:43.073767 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.073416    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:19:43.076565 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:43.076544    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab073a4e_177f_48c0_9091_fb355d8bce3f.slice/crio-382be86a438c003a02e83e76b531dabb8ff74f8931d79a507afcde8c90e64c7d WatchSource:0}: Error finding container 382be86a438c003a02e83e76b531dabb8ff74f8931d79a507afcde8c90e64c7d: Status 404 returned error can't find the container with id 382be86a438c003a02e83e76b531dabb8ff74f8931d79a507afcde8c90e64c7d
Sep 09 12:19:43.079724 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:43.079704    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aabdb32_70b9_4bbf_a9ca_b15d76924dc7.slice/crio-315b233d555cebb3da6c57ec4c51f50b5e4675735fdc465421195e1ef982e344 WatchSource:0}: Error finding container 315b233d555cebb3da6c57ec4c51f50b5e4675735fdc465421195e1ef982e344: Status 404 returned error can't find the container with id 315b233d555cebb3da6c57ec4c51f50b5e4675735fdc465421195e1ef982e344
Sep 09 12:19:43.084721 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:19:43.084699    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a0ff7f0_303a_45c2_b0f4_8081800ab66f.slice/crio-dd2410f21dada8f6980405f3e314a3f679be117772b949189e472833a00c07de WatchSource:0}: Error finding container dd2410f21dada8f6980405f3e314a3f679be117772b949189e472833a00c07de: Status 404 returned error can't find the container with id dd2410f21dada8f6980405f3e314a3f679be117772b949189e472833a00c07de
Sep 09 12:19:43.434753 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.434669    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2bxd\" (UniqueName: \"kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd\") pod \"network-check-target-b9h5d\" (UID: \"e2d11583-88d9-42e6-8de7-48df91ba8254\") " pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:43.434753 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.434725    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:43.434995 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.434778    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs\") pod \"network-metrics-daemon-4kv6h\" (UID: \"5fa079e3-edaf-4746-93c9-fa89298bbb94\") " pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:43.434995 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:43.434877    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:19:43.434995 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:43.434896    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:43.434995 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:43.434898    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:19:43.434995 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:43.434915    2582 projected.go:196] Error preparing data for projected volume kube-api-access-z2bxd for pod openshift-network-diagnostics/network-check-target-b9h5d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:43.434995 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:43.434954    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs podName:5fa079e3-edaf-4746-93c9-fa89298bbb94 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:44.43493303 +0000 UTC m=+4.131498161 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs") pod "network-metrics-daemon-4kv6h" (UID: "5fa079e3-edaf-4746-93c9-fa89298bbb94") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:43.434995 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:43.434974    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd podName:e2d11583-88d9-42e6-8de7-48df91ba8254 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:44.43496396 +0000 UTC m=+4.131529094 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-z2bxd" (UniqueName: "kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd") pod "network-check-target-b9h5d" (UID: "e2d11583-88d9-42e6-8de7-48df91ba8254") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:43.435365 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:43.435025    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:43.435365 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:43.435065    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret podName:cb9acdab-04f6-47b1-9bcd-d24246dcf52a nodeName:}" failed. No retries permitted until 2026-09-09 12:19:44.435050221 +0000 UTC m=+4.131615354 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret") pod "global-pull-secret-syncer-lxbw5" (UID: "cb9acdab-04f6-47b1-9bcd-d24246dcf52a") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:43.776784 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.776676    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 12:14:42 +0000 UTC" deadline="2028-05-22 01:06:57.255542862 +0000 UTC"
Sep 09 12:19:43.776784 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.776712    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14892h47m13.478835486s"
Sep 09 12:19:43.830241 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.830208    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bjqck" event={"ID":"ae8eed86-8b04-4c9a-ab62-4ea97a34849d","Type":"ContainerStarted","Data":"29e05b0358ca3047244eeada5e8b0863cbd8c69b2392b1219c2a5d2cf4766df0"}
Sep 09 12:19:43.831868 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.831827    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8zf7j" event={"ID":"25a2a8b0-4d6b-4171-b470-7fd9b57a16e8","Type":"ContainerStarted","Data":"a97a08b5669fe50e77680f824585c443ae200140c330c9f725e065c119d6f87b"}
Sep 09 12:19:43.835660 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.835634    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8tjr5" event={"ID":"18511945-5b00-4ea7-8c23-598b3f3de208","Type":"ContainerStarted","Data":"2ce6561755298fabb6219bb9ea891343d8511e63b69f0b7f2b92c8b601e1d2ba"}
Sep 09 12:19:43.837698 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.837673    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb" event={"ID":"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c","Type":"ContainerStarted","Data":"d3306927199d4301cec45f34db5371837ac897c3c7ac645bf520619578902138"}
Sep 09 12:19:43.839755 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.839611    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tqtbz" event={"ID":"5aabdb32-70b9-4bbf-a9ca-b15d76924dc7","Type":"ContainerStarted","Data":"315b233d555cebb3da6c57ec4c51f50b5e4675735fdc465421195e1ef982e344"}
Sep 09 12:19:43.841189 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.841164    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vjlwf" event={"ID":"141b4693-a55b-4c1d-89eb-f19b26a9e1f5","Type":"ContainerStarted","Data":"707f7fab9ea0ca6e101141af4c3d62f7fc892d4e539722734813bd9e6ca4ee48"}
Sep 09 12:19:43.843176 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.843151    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw" event={"ID":"8a0ff7f0-303a-45c2-b0f4-8081800ab66f","Type":"ContainerStarted","Data":"dd2410f21dada8f6980405f3e314a3f679be117772b949189e472833a00c07de"}
Sep 09 12:19:43.844657 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.844632    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pd4tx" event={"ID":"ab073a4e-177f-48c0-9091-fb355d8bce3f","Type":"ContainerStarted","Data":"382be86a438c003a02e83e76b531dabb8ff74f8931d79a507afcde8c90e64c7d"}
Sep 09 12:19:43.846344 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:43.846321    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dpwzk" event={"ID":"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c","Type":"ContainerStarted","Data":"d747c932ae89ca698f759872b4c9f11979a58396517ae42abb3dd74b859c4c8d"}
Sep 09 12:19:44.442987 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:44.442951    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2bxd\" (UniqueName: \"kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd\") pod \"network-check-target-b9h5d\" (UID: \"e2d11583-88d9-42e6-8de7-48df91ba8254\") " pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:44.443154 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:44.443008    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:44.443154 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:44.443054    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs\") pod \"network-metrics-daemon-4kv6h\" (UID: \"5fa079e3-edaf-4746-93c9-fa89298bbb94\") " pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:44.443264 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:44.443181    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:44.443264 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:44.443239    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs podName:5fa079e3-edaf-4746-93c9-fa89298bbb94 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:46.443221783 +0000 UTC m=+6.139786931 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs") pod "network-metrics-daemon-4kv6h" (UID: "5fa079e3-edaf-4746-93c9-fa89298bbb94") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:44.443389 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:44.443321    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:19:44.443389 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:44.443335    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:19:44.443389 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:44.443350    2582 projected.go:196] Error preparing data for projected volume kube-api-access-z2bxd for pod openshift-network-diagnostics/network-check-target-b9h5d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:44.443528 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:44.443388    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd podName:e2d11583-88d9-42e6-8de7-48df91ba8254 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:46.443376949 +0000 UTC m=+6.139942084 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-z2bxd" (UniqueName: "kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd") pod "network-check-target-b9h5d" (UID: "e2d11583-88d9-42e6-8de7-48df91ba8254") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:44.443528 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:44.443451    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:44.443528 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:44.443483    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret podName:cb9acdab-04f6-47b1-9bcd-d24246dcf52a nodeName:}" failed. No retries permitted until 2026-09-09 12:19:46.443472147 +0000 UTC m=+6.140037292 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret") pod "global-pull-secret-syncer-lxbw5" (UID: "cb9acdab-04f6-47b1-9bcd-d24246dcf52a") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:44.777895 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:44.777790    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 12:14:42 +0000 UTC" deadline="2028-05-04 07:53:00.852500078 +0000 UTC"
Sep 09 12:19:44.777895 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:44.777823    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14467h33m16.074681579s"
Sep 09 12:19:44.818997 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:44.818963    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:44.818997 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:44.818986    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:44.819195 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:44.819082    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:19:44.819195 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:44.819133    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:44.819299 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:44.819227    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:19:44.819354 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:44.819316    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:19:45.870612 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:45.869667    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal" event={"ID":"0512c5589b44ea46a23646a68804fa89","Type":"ContainerStarted","Data":"46b92780064a226047e42cdfd167b1df925629e33d14a2a265c92eba69f0e46e"}
Sep 09 12:19:45.886175 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:45.886090    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal" podStartSLOduration=3.886070364 podStartE2EDuration="3.886070364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:19:45.885331064 +0000 UTC m=+5.581896225" watchObservedRunningTime="2026-09-09 12:19:45.886070364 +0000 UTC m=+5.582635520"
Sep 09 12:19:46.461539 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:46.461506    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs\") pod \"network-metrics-daemon-4kv6h\" (UID: \"5fa079e3-edaf-4746-93c9-fa89298bbb94\") " pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:46.461731 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:46.461570    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2bxd\" (UniqueName: \"kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd\") pod \"network-check-target-b9h5d\" (UID: \"e2d11583-88d9-42e6-8de7-48df91ba8254\") " pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:46.461731 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:46.461608    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:46.461872 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:46.461733    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:46.461872 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:46.461804    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret podName:cb9acdab-04f6-47b1-9bcd-d24246dcf52a nodeName:}" failed. No retries permitted until 2026-09-09 12:19:50.461785816 +0000 UTC m=+10.158350950 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret") pod "global-pull-secret-syncer-lxbw5" (UID: "cb9acdab-04f6-47b1-9bcd-d24246dcf52a") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:46.462304 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:46.462284    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:46.462388 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:46.462340    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs podName:5fa079e3-edaf-4746-93c9-fa89298bbb94 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:50.462324963 +0000 UTC m=+10.158890092 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs") pod "network-metrics-daemon-4kv6h" (UID: "5fa079e3-edaf-4746-93c9-fa89298bbb94") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:46.462450 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:46.462418    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:19:46.462450 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:46.462432    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:19:46.462450 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:46.462443    2582 projected.go:196] Error preparing data for projected volume kube-api-access-z2bxd for pod openshift-network-diagnostics/network-check-target-b9h5d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:46.462607 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:46.462478    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd podName:e2d11583-88d9-42e6-8de7-48df91ba8254 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:50.46246652 +0000 UTC m=+10.159031653 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-z2bxd" (UniqueName: "kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd") pod "network-check-target-b9h5d" (UID: "e2d11583-88d9-42e6-8de7-48df91ba8254") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:46.822073 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:46.821210    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:46.822073 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:46.821321    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:19:46.822073 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:46.821683    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:46.822073 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:46.821784    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:19:46.822073 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:46.821875    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:46.822073 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:46.821950    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:19:46.874182 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:46.874143    2582 generic.go:356] "Generic (PLEG): container finished" podID="6c711764bbcbf3000f1d5e41efdec0f5" containerID="894bbfd0520d78a86e8e88602f89cff50fea1f058c3440c9df869690eb125cc8" exitCode=0
Sep 09 12:19:46.874651 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:46.874620    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal" event={"ID":"6c711764bbcbf3000f1d5e41efdec0f5","Type":"ContainerDied","Data":"894bbfd0520d78a86e8e88602f89cff50fea1f058c3440c9df869690eb125cc8"}
Sep 09 12:19:48.819312 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:48.819241    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:48.819735 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:48.819363    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:19:48.819735 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:48.819681    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:48.819881 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:48.819753    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:19:48.819881 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:48.819789    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:48.819881 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:48.819840    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:19:50.493108 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:50.493063    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs\") pod \"network-metrics-daemon-4kv6h\" (UID: \"5fa079e3-edaf-4746-93c9-fa89298bbb94\") " pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:50.493550 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:50.493119    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2bxd\" (UniqueName: \"kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd\") pod \"network-check-target-b9h5d\" (UID: \"e2d11583-88d9-42e6-8de7-48df91ba8254\") " pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:50.493550 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:50.493145    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:50.493550 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:50.493236    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:50.493550 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:50.493284    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret podName:cb9acdab-04f6-47b1-9bcd-d24246dcf52a nodeName:}" failed. No retries permitted until 2026-09-09 12:19:58.493272258 +0000 UTC m=+18.189837387 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret") pod "global-pull-secret-syncer-lxbw5" (UID: "cb9acdab-04f6-47b1-9bcd-d24246dcf52a") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:50.493782 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:50.493584    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:50.493782 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:50.493616    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs podName:5fa079e3-edaf-4746-93c9-fa89298bbb94 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:58.493606666 +0000 UTC m=+18.190171796 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs") pod "network-metrics-daemon-4kv6h" (UID: "5fa079e3-edaf-4746-93c9-fa89298bbb94") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:50.493782 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:50.493671    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:19:50.493782 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:50.493683    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:19:50.493782 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:50.493691    2582 projected.go:196] Error preparing data for projected volume kube-api-access-z2bxd for pod openshift-network-diagnostics/network-check-target-b9h5d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:50.493782 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:50.493713    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd podName:e2d11583-88d9-42e6-8de7-48df91ba8254 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:58.493707004 +0000 UTC m=+18.190272136 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-z2bxd" (UniqueName: "kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd") pod "network-check-target-b9h5d" (UID: "e2d11583-88d9-42e6-8de7-48df91ba8254") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:50.820069 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:50.819965    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:50.820069 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:50.819966    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:50.821069 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:50.821047    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:50.821680 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:50.821161    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:19:50.821680 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:50.821275    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:19:50.821680 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:50.821346    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:19:52.818632 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:52.818603    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:52.818632 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:52.818620    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:52.819057 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:52.818713    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:19:52.819057 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:52.818787    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:52.819057 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:52.818800    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:19:52.819057 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:52.818889    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:19:54.818988 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:54.818953    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:54.818988 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:54.818996    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:54.819464 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:54.819086    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:19:54.819464 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:54.819156    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:54.819464 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:54.819258    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:19:54.819464 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:54.819339    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:19:56.821395 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:56.821366    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:56.821836 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:56.821366    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:56.821836 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:56.821496    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:19:56.821836 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:56.821366    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:56.821836 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:56.821574    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:19:56.821836 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:56.821655    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:19:58.548964 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:58.548919    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs\") pod \"network-metrics-daemon-4kv6h\" (UID: \"5fa079e3-edaf-4746-93c9-fa89298bbb94\") " pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:58.548964 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:58.548964    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2bxd\" (UniqueName: \"kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd\") pod \"network-check-target-b9h5d\" (UID: \"e2d11583-88d9-42e6-8de7-48df91ba8254\") " pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:58.549495 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:58.549002    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:58.549495 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:58.549102    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:58.549495 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:58.549104    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:58.549495 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:58.549162    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret podName:cb9acdab-04f6-47b1-9bcd-d24246dcf52a nodeName:}" failed. No retries permitted until 2026-09-09 12:20:14.549143996 +0000 UTC m=+34.245709142 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret") pod "global-pull-secret-syncer-lxbw5" (UID: "cb9acdab-04f6-47b1-9bcd-d24246dcf52a") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:58.549495 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:58.549174    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:19:58.549495 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:58.549196    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:19:58.549495 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:58.549206    2582 projected.go:196] Error preparing data for projected volume kube-api-access-z2bxd for pod openshift-network-diagnostics/network-check-target-b9h5d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:58.549495 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:58.549177    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs podName:5fa079e3-edaf-4746-93c9-fa89298bbb94 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:14.54917008 +0000 UTC m=+34.245735211 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs") pod "network-metrics-daemon-4kv6h" (UID: "5fa079e3-edaf-4746-93c9-fa89298bbb94") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:58.549495 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:58.549256    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd podName:e2d11583-88d9-42e6-8de7-48df91ba8254 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:14.549243283 +0000 UTC m=+34.245808413 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-z2bxd" (UniqueName: "kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd") pod "network-check-target-b9h5d" (UID: "e2d11583-88d9-42e6-8de7-48df91ba8254") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:58.818524 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:58.818436    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:19:58.818524 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:58.818469    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:19:58.818715 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:58.818578    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:19:58.818715 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:19:58.818660    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:19:58.818715 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:58.818688    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:19:58.818886 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:19:58.818760    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:20:00.820209 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:00.820075    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:20:00.820209 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:00.820156    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:20:00.820209 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:00.820177    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:20:00.820770 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:00.820249    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:20:00.820770 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:00.820273    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:20:00.820770 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:00.820358    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:20:01.906433 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.906238    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw" event={"ID":"8a0ff7f0-303a-45c2-b0f4-8081800ab66f","Type":"ContainerStarted","Data":"fa7ded301a286fb507f60d3806388e61167f339ed11e24bef534e90efdfa0186"}
Sep 09 12:20:01.907212 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.906524    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw" event={"ID":"8a0ff7f0-303a-45c2-b0f4-8081800ab66f","Type":"ContainerStarted","Data":"0131a3c04450e9f2788027b36123fd19f399d8f831c7759b41d0d75b78ca4fdc"}
Sep 09 12:20:01.907212 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.906566    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw" event={"ID":"8a0ff7f0-303a-45c2-b0f4-8081800ab66f","Type":"ContainerStarted","Data":"609adcab4e1183be6732c2d1f0e51e0c996ac5668f84f3a3f20bc7affbd1452f"}
Sep 09 12:20:01.907212 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.906640    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw" event={"ID":"8a0ff7f0-303a-45c2-b0f4-8081800ab66f","Type":"ContainerStarted","Data":"208a4e3e0a6f954a1647c6e298f7263b0e8d62f3ccbb6cc32f323e25ba262eef"}
Sep 09 12:20:01.908049 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.908021    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dpwzk" event={"ID":"3f6ae89a-46c0-4adc-ac1d-9cd0e286f25c","Type":"ContainerStarted","Data":"3c6686e55f103a74c960e3672c397ad63a8114c651f09ccb89e7fa657ad9070e"}
Sep 09 12:20:01.909793 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.909767    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bjqck" event={"ID":"ae8eed86-8b04-4c9a-ab62-4ea97a34849d","Type":"ContainerStarted","Data":"9813b2abcae9acbe1506eec0bf4722d2ad811228b09619b931c816a4a5e7d409"}
Sep 09 12:20:01.911547 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.911524    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8zf7j" event={"ID":"25a2a8b0-4d6b-4171-b470-7fd9b57a16e8","Type":"ContainerStarted","Data":"4983917a25605bce62f430fc2bcf6bd1cde42c5cefe4fc8e2cdf7cd6cd41d739"}
Sep 09 12:20:01.913196 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.913174    2582 generic.go:356] "Generic (PLEG): container finished" podID="18511945-5b00-4ea7-8c23-598b3f3de208" containerID="298173acad7fa21c907a35cf21fa51c306ff6be65e1af3d5e1328da76c9ec97e" exitCode=0
Sep 09 12:20:01.913297 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.913241    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8tjr5" event={"ID":"18511945-5b00-4ea7-8c23-598b3f3de208","Type":"ContainerDied","Data":"298173acad7fa21c907a35cf21fa51c306ff6be65e1af3d5e1328da76c9ec97e"}
Sep 09 12:20:01.915032 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.915003    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb" event={"ID":"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c","Type":"ContainerStarted","Data":"383d0d8daa3abefc3a7aa8fd56e8c3af729d1b32a601d80492deee7ced24b73a"}
Sep 09 12:20:01.917161 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.916991    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal" event={"ID":"6c711764bbcbf3000f1d5e41efdec0f5","Type":"ContainerStarted","Data":"80b2d744690eb0394e507bb1f80a7e604b174bf7d277710b3c6a64125bdeb423"}
Sep 09 12:20:01.918573 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.918547    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tqtbz" event={"ID":"5aabdb32-70b9-4bbf-a9ca-b15d76924dc7","Type":"ContainerStarted","Data":"5f819ee41d0cb31ece0901462ae7ac973cc6de7e3692a1f6be24dd588821324b"}
Sep 09 12:20:01.920077 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.920057    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vjlwf" event={"ID":"141b4693-a55b-4c1d-89eb-f19b26a9e1f5","Type":"ContainerStarted","Data":"e3ef1453668bbe69b4f7c1c88f729029237bc1e997db11cece65cc88cd6658dd"}
Sep 09 12:20:01.929945 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.929904    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dpwzk" podStartSLOduration=3.348863834 podStartE2EDuration="21.929892328s" totalImagesPullingTime="18.581028494s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:01.92968021 +0000 UTC m=+21.626245362" watchObservedRunningTime="2026-09-09 12:20:01.929892328 +0000 UTC m=+21.626457479"
Sep 09 12:20:01.946491 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.946440    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tqtbz" podStartSLOduration=3.632430351 podStartE2EDuration="21.94642841s" totalImagesPullingTime="18.313998059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:01.946166305 +0000 UTC m=+21.642731464" watchObservedRunningTime="2026-09-09 12:20:01.94642841 +0000 UTC m=+21.642993562"
Sep 09 12:20:01.977653 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.977609    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bjqck" podStartSLOduration=3.655232061 podStartE2EDuration="21.977594414s" totalImagesPullingTime="18.322362353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:01.977235831 +0000 UTC m=+21.673800994" watchObservedRunningTime="2026-09-09 12:20:01.977594414 +0000 UTC m=+21.674159566"
Sep 09 12:20:01.992586 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:01.992538    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vjlwf" podStartSLOduration=3.6678125809999997 podStartE2EDuration="21.992517472s" totalImagesPullingTime="18.324704891s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:01.992477893 +0000 UTC m=+21.689043045" watchObservedRunningTime="2026-09-09 12:20:01.992517472 +0000 UTC m=+21.689082623"
Sep 09 12:20:02.007222 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:02.007187    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal" podStartSLOduration=20.007175115 podStartE2EDuration="20.007175115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:02.006885596 +0000 UTC m=+21.703450747" watchObservedRunningTime="2026-09-09 12:20:02.007175115 +0000 UTC m=+21.703740266"
Sep 09 12:20:02.819074 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:02.819043    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:20:02.819292 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:02.819059    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:20:02.819292 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:02.819170    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:20:02.819292 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:02.819164    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:20:02.819432 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:02.819287    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:20:02.819432 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:02.819398    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:20:02.926086 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:02.926006    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw" event={"ID":"8a0ff7f0-303a-45c2-b0f4-8081800ab66f","Type":"ContainerStarted","Data":"b74b0ceb5ffe5478285337fc986a011860bfd2d744e2043989e34ed1fe2f8641"}
Sep 09 12:20:02.926490 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:02.926095    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw" event={"ID":"8a0ff7f0-303a-45c2-b0f4-8081800ab66f","Type":"ContainerStarted","Data":"a9c869fc80aad22ede5518f6877c904ba10b3bc1f3994ad2f7174eabc6c8d8eb"}
Sep 09 12:20:02.928784 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:02.928756    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pd4tx" event={"ID":"ab073a4e-177f-48c0-9091-fb355d8bce3f","Type":"ContainerStarted","Data":"e13143b3a08b17b233d8dde2641c0c39c0611fb7c8bd369cd11d54f9c79ac3a7"}
Sep 09 12:20:02.944146 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:02.944099    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8zf7j" podStartSLOduration=8.720350576 podStartE2EDuration="22.944080869s" totalImagesPullingTime="14.223730293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:02.027159584 +0000 UTC m=+21.723724736" watchObservedRunningTime="2026-09-09 12:20:02.944080869 +0000 UTC m=+22.640646024"
Sep 09 12:20:03.452826 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:03.452800    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tqtbz"
Sep 09 12:20:03.633298 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:03.633268    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 12:20:03.795683 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:03.795551    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T12:20:03.633290414Z","UUID":"55c7bc86-4ece-43fd-adca-c61c93c80198","Handler":null,"Name":"","Endpoint":""}
Sep 09 12:20:03.797166 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:03.797137    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 12:20:03.797166 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:03.797171    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 12:20:03.933205 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:03.933168    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb" event={"ID":"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c","Type":"ContainerStarted","Data":"7488b1403abe37de3ea85f239914bb08c7d5e45c198184e4c9cbdea4a82bd87b"}
Sep 09 12:20:04.818828 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:04.818803    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:20:04.818947 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:04.818929    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:20:04.819006 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:04.818943    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:20:04.819063 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:04.819046    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:20:04.819111 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:04.819092    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:20:04.819178 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:04.819159    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:20:04.940053 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:04.940022    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw" event={"ID":"8a0ff7f0-303a-45c2-b0f4-8081800ab66f","Type":"ContainerStarted","Data":"b9e1676731f079d5bc32c8f359de45745efa879a65d991c3fb06f434b34f857a"}
Sep 09 12:20:04.942586 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:04.942544    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb" event={"ID":"c9d83ffe-6f27-47a3-ba20-1475f4cc8d7c","Type":"ContainerStarted","Data":"5e1f664caf1554617d4b508ebc01a500576059815e01a8cb54fed694dea61a64"}
Sep 09 12:20:04.963744 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:04.963696    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pd4tx" podStartSLOduration=10.770601173 podStartE2EDuration="24.963678039s" totalImagesPullingTime="14.193076866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:02.944714804 +0000 UTC m=+22.641279957" watchObservedRunningTime="2026-09-09 12:20:04.963678039 +0000 UTC m=+24.660243191"
Sep 09 12:20:04.964336 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:04.964301    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vbrb" podStartSLOduration=3.603172651 podStartE2EDuration="24.964293481s" totalImagesPullingTime="21.36112083s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:04.963674767 +0000 UTC m=+24.660239928" watchObservedRunningTime="2026-09-09 12:20:04.964293481 +0000 UTC m=+24.660858635"
Sep 09 12:20:06.533613 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:06.533522    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tqtbz"
Sep 09 12:20:06.534272 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:06.534252    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tqtbz"
Sep 09 12:20:06.819154 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:06.819060    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:20:06.819154 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:06.819094    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:20:06.819352 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:06.819182    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:20:06.819352 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:06.819195    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:20:06.819468 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:06.819327    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:20:06.819468 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:06.819435    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:20:06.947947 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:06.947923    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tqtbz"
Sep 09 12:20:07.953383 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:07.953171    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw" event={"ID":"8a0ff7f0-303a-45c2-b0f4-8081800ab66f","Type":"ContainerStarted","Data":"c1f96aa125623718b2f810daacfb719a789a8102ac24999edbcba435a7a4f081"}
Sep 09 12:20:07.954138 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:07.953432    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:20:07.955729 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:07.955707    2582 generic.go:356] "Generic (PLEG): container finished" podID="18511945-5b00-4ea7-8c23-598b3f3de208" containerID="1c270770804a4b0bd1abc3e1db845f7c3b604865f579d679d9fa47134081dd8d" exitCode=0
Sep 09 12:20:07.955862 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:07.955787    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8tjr5" event={"ID":"18511945-5b00-4ea7-8c23-598b3f3de208","Type":"ContainerDied","Data":"1c270770804a4b0bd1abc3e1db845f7c3b604865f579d679d9fa47134081dd8d"}
Sep 09 12:20:07.969872 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:07.969830    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:20:07.979815 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:07.979782    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw" podStartSLOduration=8.643725882 podStartE2EDuration="26.979770086s" totalImagesPullingTime="18.336044204s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:07.978769305 +0000 UTC m=+27.675334455" watchObservedRunningTime="2026-09-09 12:20:07.979770086 +0000 UTC m=+27.676335237"
Sep 09 12:20:08.819001 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:08.818969    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:20:08.819175 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:08.818970    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:20:08.819175 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:08.819105    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:20:08.819175 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:08.818970    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:20:08.819175 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:08.819149    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:20:08.819375 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:08.819205    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:20:08.959156 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:08.958718    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:20:08.959156 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:08.959095    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:20:08.977600 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:08.977575    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:20:09.201064 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:09.201033    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b9h5d"]
Sep 09 12:20:09.201235 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:09.201152    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:20:09.201283 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:09.201264    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:20:09.209259 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:09.209231    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4kv6h"]
Sep 09 12:20:09.209554 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:09.209527    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:20:09.209693 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:09.209669    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:20:09.209842 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:09.209804    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lxbw5"]
Sep 09 12:20:09.209947 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:09.209931    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:20:09.210018 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:09.210001    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:20:09.961649 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:09.961582    2582 generic.go:356] "Generic (PLEG): container finished" podID="18511945-5b00-4ea7-8c23-598b3f3de208" containerID="bc9e29d70995d1e70b5dec0da8ed330ac3db3c26c3f91070a73f9ef5fd8f0da7" exitCode=0
Sep 09 12:20:09.962024 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:09.961662    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8tjr5" event={"ID":"18511945-5b00-4ea7-8c23-598b3f3de208","Type":"ContainerDied","Data":"bc9e29d70995d1e70b5dec0da8ed330ac3db3c26c3f91070a73f9ef5fd8f0da7"}
Sep 09 12:20:10.819049 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:10.819025    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:20:10.819194 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:10.819105    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:20:10.819194 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:10.819119    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:20:10.819194 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:10.819170    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:20:10.819295 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:10.819266    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:20:10.819391 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:10.819360    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:20:11.968072 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:11.968040    2582 generic.go:356] "Generic (PLEG): container finished" podID="18511945-5b00-4ea7-8c23-598b3f3de208" containerID="af4f727e540e26dfcd7c3c700182a12a402fc7af772aebc244a9d375eb0a261e" exitCode=0
Sep 09 12:20:11.968433 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:11.968092    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8tjr5" event={"ID":"18511945-5b00-4ea7-8c23-598b3f3de208","Type":"ContainerDied","Data":"af4f727e540e26dfcd7c3c700182a12a402fc7af772aebc244a9d375eb0a261e"}
Sep 09 12:20:12.819210 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:12.818888    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:20:12.819366 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:12.818960    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:20:12.819437 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:12.819000    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:20:12.819489 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:12.819387    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4kv6h" podUID="5fa079e3-edaf-4746-93c9-fa89298bbb94"
Sep 09 12:20:12.819548 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:12.819259    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b9h5d" podUID="e2d11583-88d9-42e6-8de7-48df91ba8254"
Sep 09 12:20:12.819597 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:12.819550    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lxbw5" podUID="cb9acdab-04f6-47b1-9bcd-d24246dcf52a"
Sep 09 12:20:14.563764 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.563685    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2bxd\" (UniqueName: \"kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd\") pod \"network-check-target-b9h5d\" (UID: \"e2d11583-88d9-42e6-8de7-48df91ba8254\") " pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:20:14.563764 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.563751    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:20:14.564257 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.563804    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs\") pod \"network-metrics-daemon-4kv6h\" (UID: \"5fa079e3-edaf-4746-93c9-fa89298bbb94\") " pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:20:14.564257 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:14.563932    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:20:14.564257 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:14.563964    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:20:14.564257 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:14.563997    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs podName:5fa079e3-edaf-4746-93c9-fa89298bbb94 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:46.563977133 +0000 UTC m=+66.260542285 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs") pod "network-metrics-daemon-4kv6h" (UID: "5fa079e3-edaf-4746-93c9-fa89298bbb94") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:20:14.564257 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:14.564024    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret podName:cb9acdab-04f6-47b1-9bcd-d24246dcf52a nodeName:}" failed. No retries permitted until 2026-09-09 12:20:46.564007914 +0000 UTC m=+66.260573057 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret") pod "global-pull-secret-syncer-lxbw5" (UID: "cb9acdab-04f6-47b1-9bcd-d24246dcf52a") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:20:14.564257 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:14.564045    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:20:14.564257 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:14.564063    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:20:14.564257 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:14.564075    2582 projected.go:196] Error preparing data for projected volume kube-api-access-z2bxd for pod openshift-network-diagnostics/network-check-target-b9h5d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:20:14.564257 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:14.564134    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd podName:e2d11583-88d9-42e6-8de7-48df91ba8254 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:46.564120417 +0000 UTC m=+66.260685547 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-z2bxd" (UniqueName: "kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd") pod "network-check-target-b9h5d" (UID: "e2d11583-88d9-42e6-8de7-48df91ba8254") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:20:14.599763 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.599733    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeReady"
Sep 09 12:20:14.599920 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.599870    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 12:20:14.656911 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.656881    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lq49k"]
Sep 09 12:20:14.677761 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.677737    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-z8n7s"]
Sep 09 12:20:14.677917 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.677889    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:14.679946 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.679918    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 12:20:14.680066 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.679946    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 12:20:14.680066 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.679958    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hsrxf\""
Sep 09 12:20:14.691699 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.691677    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84d6d7cf9f-96n79"]
Sep 09 12:20:14.692046 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.691929    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z8n7s"
Sep 09 12:20:14.693720 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.693700    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kls2h\""
Sep 09 12:20:14.694066 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.694038    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 12:20:14.694159 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.694142    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 12:20:14.694353 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.694312    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 12:20:14.709833 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.709807    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z8n7s"]
Sep 09 12:20:14.709956 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.709837    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lq49k"]
Sep 09 12:20:14.709956 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.709895    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84d6d7cf9f-96n79"]
Sep 09 12:20:14.710047 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.709982    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.712267 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.712246    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 12:20:14.712365 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.712247    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 12:20:14.712365 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.712248    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 12:20:14.713209 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.713190    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bswb5\""
Sep 09 12:20:14.719425 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.719407    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 12:20:14.765497 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.765474    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72bee612-067e-4654-9095-41e974f65516-tmp-dir\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:14.765580 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.765500    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-installation-pull-secrets\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.765580 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.765517    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/72bee612-067e-4654-9095-41e974f65516-config-volume\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:14.765580 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.765531    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:14.765580 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.765551    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-trusted-ca\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.765580 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.765572    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqjs8\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-kube-api-access-mqjs8\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.765775 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.765640    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-bound-sa-token\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.765775 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.765672    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-image-registry-private-configuration\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.765775 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.765732    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-ca-trust-extracted\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.765906 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.765811    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp9bw\" (UniqueName: \"kubernetes.io/projected/72bee612-067e-4654-9095-41e974f65516-kube-api-access-dp9bw\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:14.765906 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.765836    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.765986 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.765903    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnnml\" (UniqueName: \"kubernetes.io/projected/9e6412b5-162d-4538-8533-8806bbe0e8c9-kube-api-access-rnnml\") pod \"ingress-canary-z8n7s\" (UID: \"9e6412b5-162d-4538-8533-8806bbe0e8c9\") " pod="openshift-ingress-canary/ingress-canary-z8n7s"
Sep 09 12:20:14.765986 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.765933    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-certificates\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.766054 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.765986    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e6412b5-162d-4538-8533-8806bbe0e8c9-cert\") pod \"ingress-canary-z8n7s\" (UID: \"9e6412b5-162d-4538-8533-8806bbe0e8c9\") " pod="openshift-ingress-canary/ingress-canary-z8n7s"
Sep 09 12:20:14.819138 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.819077    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:20:14.819236 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.819079    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:20:14.819276 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.819242    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:20:14.821743 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.821720    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 12:20:14.821832 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.821744    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 12:20:14.821832 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.821812    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wjmxh\""
Sep 09 12:20:14.822422 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.822402    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 12:20:14.822680 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.822665    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lblnd\""
Sep 09 12:20:14.823620 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.823603    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 12:20:14.867073 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867047    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp9bw\" (UniqueName: \"kubernetes.io/projected/72bee612-067e-4654-9095-41e974f65516-kube-api-access-dp9bw\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:14.867174 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867081    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.867174 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867103    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnnml\" (UniqueName: \"kubernetes.io/projected/9e6412b5-162d-4538-8533-8806bbe0e8c9-kube-api-access-rnnml\") pod \"ingress-canary-z8n7s\" (UID: \"9e6412b5-162d-4538-8533-8806bbe0e8c9\") " pod="openshift-ingress-canary/ingress-canary-z8n7s"
Sep 09 12:20:14.867276 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:14.867217    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:20:14.867276 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:14.867237    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84d6d7cf9f-96n79: secret "image-registry-tls" not found
Sep 09 12:20:14.867276 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867234    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-certificates\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.867417 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867287    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e6412b5-162d-4538-8533-8806bbe0e8c9-cert\") pod \"ingress-canary-z8n7s\" (UID: \"9e6412b5-162d-4538-8533-8806bbe0e8c9\") " pod="openshift-ingress-canary/ingress-canary-z8n7s"
Sep 09 12:20:14.867417 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:14.867305    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls podName:68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c nodeName:}" failed. No retries permitted until 2026-09-09 12:20:15.367283977 +0000 UTC m=+35.063849109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls") pod "image-registry-84d6d7cf9f-96n79" (UID: "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c") : secret "image-registry-tls" not found
Sep 09 12:20:14.867417 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867362    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72bee612-067e-4654-9095-41e974f65516-tmp-dir\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:14.867417 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867388    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-installation-pull-secrets\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.867417 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867406    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/72bee612-067e-4654-9095-41e974f65516-config-volume\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:14.867417 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867420    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:14.867697 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867434    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-trusted-ca\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.867697 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867450    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqjs8\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-kube-api-access-mqjs8\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.867697 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867477    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-bound-sa-token\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.867697 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867494    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-image-registry-private-configuration\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.867697 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867509    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-ca-trust-extracted\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.867697 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867618    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72bee612-067e-4654-9095-41e974f65516-tmp-dir\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:14.868012 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867748    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-ca-trust-extracted\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.868012 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:14.867805    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:20:14.868012 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:14.867838    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls podName:72bee612-067e-4654-9095-41e974f65516 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:15.367824582 +0000 UTC m=+35.064389725 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls") pod "dns-default-lq49k" (UID: "72bee612-067e-4654-9095-41e974f65516") : secret "dns-default-metrics-tls" not found
Sep 09 12:20:14.868012 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.867902    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-certificates\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.868282 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.868260    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/72bee612-067e-4654-9095-41e974f65516-config-volume\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:14.868562 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.868537    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-trusted-ca\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.872150 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.872127    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-image-registry-private-configuration\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.872252 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.872135    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e6412b5-162d-4538-8533-8806bbe0e8c9-cert\") pod \"ingress-canary-z8n7s\" (UID: \"9e6412b5-162d-4538-8533-8806bbe0e8c9\") " pod="openshift-ingress-canary/ingress-canary-z8n7s"
Sep 09 12:20:14.872745 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.872717    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-installation-pull-secrets\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.875979 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.875933    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-bound-sa-token\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:14.876133 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.876111    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnnml\" (UniqueName: \"kubernetes.io/projected/9e6412b5-162d-4538-8533-8806bbe0e8c9-kube-api-access-rnnml\") pod \"ingress-canary-z8n7s\" (UID: \"9e6412b5-162d-4538-8533-8806bbe0e8c9\") " pod="openshift-ingress-canary/ingress-canary-z8n7s"
Sep 09 12:20:14.876506 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.876483    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp9bw\" (UniqueName: \"kubernetes.io/projected/72bee612-067e-4654-9095-41e974f65516-kube-api-access-dp9bw\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:14.876981 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:14.876962    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqjs8\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-kube-api-access-mqjs8\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:15.002613 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:15.002588    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z8n7s"
Sep 09 12:20:15.370368 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:15.370337    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:15.370529 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:15.370393    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:15.370529 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:15.370498    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:20:15.370529 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:15.370506    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:20:15.370529 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:15.370522    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84d6d7cf9f-96n79: secret "image-registry-tls" not found
Sep 09 12:20:15.370694 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:15.370564    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls podName:72bee612-067e-4654-9095-41e974f65516 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:16.370548957 +0000 UTC m=+36.067114086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls") pod "dns-default-lq49k" (UID: "72bee612-067e-4654-9095-41e974f65516") : secret "dns-default-metrics-tls" not found
Sep 09 12:20:15.370694 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:15.370579    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls podName:68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c nodeName:}" failed. No retries permitted until 2026-09-09 12:20:16.370572476 +0000 UTC m=+36.067137605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls") pod "image-registry-84d6d7cf9f-96n79" (UID: "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c") : secret "image-registry-tls" not found
Sep 09 12:20:16.350629 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:16.350591    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z8n7s"]
Sep 09 12:20:16.355274 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:20:16.355252    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e6412b5_162d_4538_8533_8806bbe0e8c9.slice/crio-3dfde0c9ffcf87fc9f27e51bd7b5cae03b32b931fc22a08de3aa0160c8afeb0f WatchSource:0}: Error finding container 3dfde0c9ffcf87fc9f27e51bd7b5cae03b32b931fc22a08de3aa0160c8afeb0f: Status 404 returned error can't find the container with id 3dfde0c9ffcf87fc9f27e51bd7b5cae03b32b931fc22a08de3aa0160c8afeb0f
Sep 09 12:20:16.377872 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:16.377823    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:16.377998 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:16.377941    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:16.377998 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:16.377973    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:20:16.378093 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:16.378032    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls podName:72bee612-067e-4654-9095-41e974f65516 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:18.378015594 +0000 UTC m=+38.074580743 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls") pod "dns-default-lq49k" (UID: "72bee612-067e-4654-9095-41e974f65516") : secret "dns-default-metrics-tls" not found
Sep 09 12:20:16.378093 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:16.378054    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:20:16.378093 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:16.378064    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84d6d7cf9f-96n79: secret "image-registry-tls" not found
Sep 09 12:20:16.378242 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:16.378096    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls podName:68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c nodeName:}" failed. No retries permitted until 2026-09-09 12:20:18.378084861 +0000 UTC m=+38.074650006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls") pod "image-registry-84d6d7cf9f-96n79" (UID: "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c") : secret "image-registry-tls" not found
Sep 09 12:20:16.982523 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:16.982307    2582 generic.go:356] "Generic (PLEG): container finished" podID="18511945-5b00-4ea7-8c23-598b3f3de208" containerID="7a6848f76e8739d1c927a091cad0a9c084ee393c71add5e6033c9bdb42e5ea66" exitCode=0
Sep 09 12:20:16.982653 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:16.982380    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8tjr5" event={"ID":"18511945-5b00-4ea7-8c23-598b3f3de208","Type":"ContainerDied","Data":"7a6848f76e8739d1c927a091cad0a9c084ee393c71add5e6033c9bdb42e5ea66"}
Sep 09 12:20:16.983695 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:16.983674    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z8n7s" event={"ID":"9e6412b5-162d-4538-8533-8806bbe0e8c9","Type":"ContainerStarted","Data":"3dfde0c9ffcf87fc9f27e51bd7b5cae03b32b931fc22a08de3aa0160c8afeb0f"}
Sep 09 12:20:17.989011 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:17.988973    2582 generic.go:356] "Generic (PLEG): container finished" podID="18511945-5b00-4ea7-8c23-598b3f3de208" containerID="8a4d131c3b0790042cb2a1f7752417500e97efcc06cdb93c83bffc4e89e63309" exitCode=0
Sep 09 12:20:17.989372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:17.989025    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8tjr5" event={"ID":"18511945-5b00-4ea7-8c23-598b3f3de208","Type":"ContainerDied","Data":"8a4d131c3b0790042cb2a1f7752417500e97efcc06cdb93c83bffc4e89e63309"}
Sep 09 12:20:18.392295 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:18.392242    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:18.392460 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:18.392331    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:18.392460 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:18.392400    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:20:18.392460 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:18.392434    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:20:18.392460 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:18.392445    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84d6d7cf9f-96n79: secret "image-registry-tls" not found
Sep 09 12:20:18.392600 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:18.392476    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls podName:72bee612-067e-4654-9095-41e974f65516 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:22.392455013 +0000 UTC m=+42.089020143 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls") pod "dns-default-lq49k" (UID: "72bee612-067e-4654-9095-41e974f65516") : secret "dns-default-metrics-tls" not found
Sep 09 12:20:18.392600 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:18.392491    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls podName:68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c nodeName:}" failed. No retries permitted until 2026-09-09 12:20:22.392484886 +0000 UTC m=+42.089050015 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls") pod "image-registry-84d6d7cf9f-96n79" (UID: "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c") : secret "image-registry-tls" not found
Sep 09 12:20:18.994592 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:18.994560    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8tjr5" event={"ID":"18511945-5b00-4ea7-8c23-598b3f3de208","Type":"ContainerStarted","Data":"33ae310651ef2febcbbcb46abd00a896e9c2ee49c6a8857d0aab795367c994e5"}
Sep 09 12:20:19.017124 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:19.017041    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8tjr5" podStartSLOduration=7.999313443 podStartE2EDuration="39.017019499s" totalImagesPullingTime="31.017706056s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:19.016887372 +0000 UTC m=+38.713452522" watchObservedRunningTime="2026-09-09 12:20:19.017019499 +0000 UTC m=+38.713584649"
Sep 09 12:20:19.998647 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:19.998610    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z8n7s" event={"ID":"9e6412b5-162d-4538-8533-8806bbe0e8c9","Type":"ContainerStarted","Data":"b89f6e38e8f74c5ecd88fb62a4fe4a97045c40642bde4d7d99095e43c91b5883"}
Sep 09 12:20:22.417705 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:22.417666    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:22.418077 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:22.417733    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:22.418077 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:22.417811    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:20:22.418077 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:22.417826    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:20:22.418077 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:22.417832    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84d6d7cf9f-96n79: secret "image-registry-tls" not found
Sep 09 12:20:22.418077 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:22.417908    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls podName:68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c nodeName:}" failed. No retries permitted until 2026-09-09 12:20:30.417894156 +0000 UTC m=+50.114459286 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls") pod "image-registry-84d6d7cf9f-96n79" (UID: "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c") : secret "image-registry-tls" not found
Sep 09 12:20:22.418077 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:22.417921    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls podName:72bee612-067e-4654-9095-41e974f65516 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:30.417915082 +0000 UTC m=+50.114480211 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls") pod "dns-default-lq49k" (UID: "72bee612-067e-4654-9095-41e974f65516") : secret "dns-default-metrics-tls" not found
Sep 09 12:20:25.174162 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.174108    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-z8n7s" podStartSLOduration=8.198717417 podStartE2EDuration="11.174092473s" totalImagesPullingTime="2.975375056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:20.014804563 +0000 UTC m=+39.711369716" watchObservedRunningTime="2026-09-09 12:20:25.174092473 +0000 UTC m=+44.870657630"
Sep 09 12:20:25.174701 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.174230    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gf8fx"]
Sep 09 12:20:25.204023 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.203996    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gf8fx"]
Sep 09 12:20:25.204130 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.204087    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.206232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.206212    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 12:20:25.206328 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.206212    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 12:20:25.206581 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.206553    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 12:20:25.206677 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.206589    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tndcj\""
Sep 09 12:20:25.206677 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.206639    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 12:20:25.339340 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.339320    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-crio-socket\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.339442 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.339344    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-data-volume\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.339442 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.339384    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.339526 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.339451    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.339526 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.339493    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x5pz\" (UniqueName: \"kubernetes.io/projected/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-api-access-8x5pz\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.439774 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.439722    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-crio-socket\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.439774 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.439749    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-data-volume\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.439911 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.439801    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.439911 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.439837    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.439911 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.439899    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x5pz\" (UniqueName: \"kubernetes.io/projected/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-api-access-8x5pz\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.440025 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:25.439928    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:20:25.440025 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.439961    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-crio-socket\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.440025 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:25.440006    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-rbac-proxy-cm podName:8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:25.939991767 +0000 UTC m=+45.636556900 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gf8fx" (UID: "8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4") : configmap "kube-rbac-proxy" not found
Sep 09 12:20:25.440132 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:25.440030    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:25.440132 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.440074    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-data-volume\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.440132 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:25.440093    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls podName:8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:25.9400785 +0000 UTC m=+45.636643653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gf8fx" (UID: "8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:25.451498 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.451474    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x5pz\" (UniqueName: \"kubernetes.io/projected/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-api-access-8x5pz\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.943138 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.943104    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.943313 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:25.943159    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:25.943313 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:25.943218    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:20:25.943313 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:25.943281    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-rbac-proxy-cm podName:8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:26.943266501 +0000 UTC m=+46.639831630 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gf8fx" (UID: "8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4") : configmap "kube-rbac-proxy" not found
Sep 09 12:20:25.943463 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:25.943313    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:25.943463 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:25.943382    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls podName:8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:26.943365329 +0000 UTC m=+46.639930476 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gf8fx" (UID: "8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:26.938883 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:26.938838    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8zf7j_25a2a8b0-4d6b-4171-b470-7fd9b57a16e8/dns-node-resolver/0.log"
Sep 09 12:20:26.950161 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:26.950138    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:26.950257 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:26.950176    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:26.950314 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:26.950273    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:20:26.950314 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:26.950280    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:26.950397 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:26.950327    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls podName:8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:28.950315704 +0000 UTC m=+48.646880833 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gf8fx" (UID: "8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:26.950397 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:26.950349    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-rbac-proxy-cm podName:8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:28.950331669 +0000 UTC m=+48.646896802 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gf8fx" (UID: "8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4") : configmap "kube-rbac-proxy" not found
Sep 09 12:20:27.939188 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:27.939158    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bjqck_ae8eed86-8b04-4c9a-ab62-4ea97a34849d/node-ca/0.log"
Sep 09 12:20:28.967467 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:28.967439    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:28.967795 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:28.967483    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:28.967795 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:28.967568    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:20:28.967795 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:28.967598    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:28.967795 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:28.967647    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls podName:8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:32.967629836 +0000 UTC m=+52.664194981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gf8fx" (UID: "8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:28.967795 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:28.967661    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-rbac-proxy-cm podName:8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:32.967655143 +0000 UTC m=+52.664220272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gf8fx" (UID: "8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4") : configmap "kube-rbac-proxy" not found
Sep 09 12:20:29.139474 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:29.139447    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z8n7s_9e6412b5-162d-4538-8533-8806bbe0e8c9/serve-healthcheck-canary/0.log"
Sep 09 12:20:30.481823 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:30.481781    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:30.482349 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:30.481904    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:30.482349 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:30.482012    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:20:30.482349 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:30.482019    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:20:30.482349 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:30.482104    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls podName:72bee612-067e-4654-9095-41e974f65516 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:46.482081214 +0000 UTC m=+66.178646344 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls") pod "dns-default-lq49k" (UID: "72bee612-067e-4654-9095-41e974f65516") : secret "dns-default-metrics-tls" not found
Sep 09 12:20:30.482349 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:30.482024    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84d6d7cf9f-96n79: secret "image-registry-tls" not found
Sep 09 12:20:30.482349 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:30.482167    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls podName:68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c nodeName:}" failed. No retries permitted until 2026-09-09 12:20:46.482154204 +0000 UTC m=+66.178719335 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls") pod "image-registry-84d6d7cf9f-96n79" (UID: "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c") : secret "image-registry-tls" not found
Sep 09 12:20:33.005392 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:33.005363    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:33.005741 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:33.005403    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:33.005741 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:33.005501    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:33.005741 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:20:33.005561    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls podName:8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:41.005545424 +0000 UTC m=+60.702110570 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gf8fx" (UID: "8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:33.005879 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:33.005833    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:40.974201 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:40.974175    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x9vqw"
Sep 09 12:20:41.069869 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:41.069820    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:41.071968 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:41.071945    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gf8fx\" (UID: \"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4\") " pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:41.115174 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:41.115152    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tndcj\""
Sep 09 12:20:41.123889 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:41.123869    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gf8fx"
Sep 09 12:20:41.392627 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:41.392595    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gf8fx"]
Sep 09 12:20:41.395698 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:20:41.395677    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ea0d14c_1bab_4d7d_b951_8e6f0b46c0e4.slice/crio-141df2bf838c880a09a0759ac724849adf2fd85f8ba68051b44f5b680b528e71 WatchSource:0}: Error finding container 141df2bf838c880a09a0759ac724849adf2fd85f8ba68051b44f5b680b528e71: Status 404 returned error can't find the container with id 141df2bf838c880a09a0759ac724849adf2fd85f8ba68051b44f5b680b528e71
Sep 09 12:20:42.050351 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:42.050311    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gf8fx" event={"ID":"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4","Type":"ContainerStarted","Data":"2a4cf7b3c8ea170585f879812b7f6452368c3951f31e0bbe1ff28eb94e7f07e7"}
Sep 09 12:20:42.050775 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:42.050366    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gf8fx" event={"ID":"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4","Type":"ContainerStarted","Data":"141df2bf838c880a09a0759ac724849adf2fd85f8ba68051b44f5b680b528e71"}
Sep 09 12:20:43.055528 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:43.055470    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gf8fx" event={"ID":"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4","Type":"ContainerStarted","Data":"b60776ddf50a674130c80b93505379edbc969dcbe66de238b6e8e03f36566c54"}
Sep 09 12:20:44.059754 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:44.059666    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gf8fx" event={"ID":"8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4","Type":"ContainerStarted","Data":"ee23c7b8cc8ca90a91ef35f345ca1b4b6998e2db4ffb896b80d98b6165aa4ecd"}
Sep 09 12:20:44.079158 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:44.079081    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gf8fx" podStartSLOduration=16.824742687 podStartE2EDuration="19.079066555s" totalImagesPullingTime="2.254323868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:20:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:44.077686056 +0000 UTC m=+63.774251208" watchObservedRunningTime="2026-09-09 12:20:44.079066555 +0000 UTC m=+63.775631706"
Sep 09 12:20:46.509328 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.509293    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:46.509705 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.509348    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:46.511702 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.511681    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/72bee612-067e-4654-9095-41e974f65516-metrics-tls\") pod \"dns-default-lq49k\" (UID: \"72bee612-067e-4654-9095-41e974f65516\") " pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:46.511900 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.511886    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls\") pod \"image-registry-84d6d7cf9f-96n79\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") " pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:46.522893 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.522875    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bswb5\""
Sep 09 12:20:46.531755 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.531742    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:46.610433 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.610404    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs\") pod \"network-metrics-daemon-4kv6h\" (UID: \"5fa079e3-edaf-4746-93c9-fa89298bbb94\") " pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:20:46.610566 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.610465    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2bxd\" (UniqueName: \"kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd\") pod \"network-check-target-b9h5d\" (UID: \"e2d11583-88d9-42e6-8de7-48df91ba8254\") " pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:20:46.610566 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.610500    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:20:46.612803 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.612784    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 12:20:46.612869 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.612802    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 12:20:46.612869 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.612814    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 12:20:46.623018 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.622997    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 12:20:46.623591 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.623567    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fa079e3-edaf-4746-93c9-fa89298bbb94-metrics-certs\") pod \"network-metrics-daemon-4kv6h\" (UID: \"5fa079e3-edaf-4746-93c9-fa89298bbb94\") " pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:20:46.623680 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.623610    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb9acdab-04f6-47b1-9bcd-d24246dcf52a-original-pull-secret\") pod \"global-pull-secret-syncer-lxbw5\" (UID: \"cb9acdab-04f6-47b1-9bcd-d24246dcf52a\") " pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:20:46.632049 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.632032    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wjmxh\""
Sep 09 12:20:46.633534 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.633518    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2bxd\" (UniqueName: \"kubernetes.io/projected/e2d11583-88d9-42e6-8de7-48df91ba8254-kube-api-access-z2bxd\") pod \"network-check-target-b9h5d\" (UID: \"e2d11583-88d9-42e6-8de7-48df91ba8254\") " pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:20:46.639075 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.639061    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lblnd\""
Sep 09 12:20:46.641378 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.641366    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4kv6h"
Sep 09 12:20:46.642534 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.642523    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lxbw5"
Sep 09 12:20:46.648117 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.648096    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:20:46.789871 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.789790    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hsrxf\""
Sep 09 12:20:46.798197 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.798181    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:46.802906 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.802873    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84d6d7cf9f-96n79"]
Sep 09 12:20:46.807685 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:20:46.807555    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68e1f6c9_b2ee_43cc_ac2b_cc3fc01c846c.slice/crio-6e9878f18d816c2e1c62528600a70caba1262bccdba2d479c5f9441c29bfe990 WatchSource:0}: Error finding container 6e9878f18d816c2e1c62528600a70caba1262bccdba2d479c5f9441c29bfe990: Status 404 returned error can't find the container with id 6e9878f18d816c2e1c62528600a70caba1262bccdba2d479c5f9441c29bfe990
Sep 09 12:20:46.912995 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.912967    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lxbw5"]
Sep 09 12:20:46.916007 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.915976    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4kv6h"]
Sep 09 12:20:46.916439 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:20:46.916417    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb9acdab_04f6_47b1_9bcd_d24246dcf52a.slice/crio-bba1ec86315fef1120a7636840abc069e717fb084ee6e3d6257e1f3dd309141b WatchSource:0}: Error finding container bba1ec86315fef1120a7636840abc069e717fb084ee6e3d6257e1f3dd309141b: Status 404 returned error can't find the container with id bba1ec86315fef1120a7636840abc069e717fb084ee6e3d6257e1f3dd309141b
Sep 09 12:20:46.919139 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:20:46.919117    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fa079e3_edaf_4746_93c9_fa89298bbb94.slice/crio-163491dea9e49eb51e5d4fc90a9b0ec8c55f570ac9b1caf11f687460ee06794d WatchSource:0}: Error finding container 163491dea9e49eb51e5d4fc90a9b0ec8c55f570ac9b1caf11f687460ee06794d: Status 404 returned error can't find the container with id 163491dea9e49eb51e5d4fc90a9b0ec8c55f570ac9b1caf11f687460ee06794d
Sep 09 12:20:46.929883 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:46.929509    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b9h5d"]
Sep 09 12:20:46.934247 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:20:46.934220    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d11583_88d9_42e6_8de7_48df91ba8254.slice/crio-5e540c7958d37c8d1170c45716b94b1c74a2986b14acfaa92c47eeae00a6defa WatchSource:0}: Error finding container 5e540c7958d37c8d1170c45716b94b1c74a2986b14acfaa92c47eeae00a6defa: Status 404 returned error can't find the container with id 5e540c7958d37c8d1170c45716b94b1c74a2986b14acfaa92c47eeae00a6defa
Sep 09 12:20:47.067913 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:47.067805    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4kv6h" event={"ID":"5fa079e3-edaf-4746-93c9-fa89298bbb94","Type":"ContainerStarted","Data":"163491dea9e49eb51e5d4fc90a9b0ec8c55f570ac9b1caf11f687460ee06794d"}
Sep 09 12:20:47.068719 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:47.068699    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lq49k"]
Sep 09 12:20:47.070042 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:47.070021    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79" event={"ID":"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c","Type":"ContainerStarted","Data":"7b1efe2af7d027efa1bbccb6f001896872ed2d917c27da2b3caf945fa8397957"}
Sep 09 12:20:47.070150 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:47.070062    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79" event={"ID":"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c","Type":"ContainerStarted","Data":"6e9878f18d816c2e1c62528600a70caba1262bccdba2d479c5f9441c29bfe990"}
Sep 09 12:20:47.070150 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:47.070117    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:20:47.071425 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:47.071403    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lxbw5" event={"ID":"cb9acdab-04f6-47b1-9bcd-d24246dcf52a","Type":"ContainerStarted","Data":"bba1ec86315fef1120a7636840abc069e717fb084ee6e3d6257e1f3dd309141b"}
Sep 09 12:20:47.072689 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:47.072662    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b9h5d" event={"ID":"e2d11583-88d9-42e6-8de7-48df91ba8254","Type":"ContainerStarted","Data":"5e540c7958d37c8d1170c45716b94b1c74a2986b14acfaa92c47eeae00a6defa"}
Sep 09 12:20:47.072961 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:20:47.072940    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72bee612_067e_4654_9095_41e974f65516.slice/crio-7cf97916dc8cf0ea1f9b9ff1100b5169845caa47cfd4a1733a830da1309595e1 WatchSource:0}: Error finding container 7cf97916dc8cf0ea1f9b9ff1100b5169845caa47cfd4a1733a830da1309595e1: Status 404 returned error can't find the container with id 7cf97916dc8cf0ea1f9b9ff1100b5169845caa47cfd4a1733a830da1309595e1
Sep 09 12:20:47.090341 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:47.090299    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79" podStartSLOduration=45.090284828 podStartE2EDuration="45.090284828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:47.089551993 +0000 UTC m=+66.786117143" watchObservedRunningTime="2026-09-09 12:20:47.090284828 +0000 UTC m=+66.786849970"
Sep 09 12:20:48.079367 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:48.079333    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lq49k" event={"ID":"72bee612-067e-4654-9095-41e974f65516","Type":"ContainerStarted","Data":"7cf97916dc8cf0ea1f9b9ff1100b5169845caa47cfd4a1733a830da1309595e1"}
Sep 09 12:20:48.081407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:48.081382    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4kv6h" event={"ID":"5fa079e3-edaf-4746-93c9-fa89298bbb94","Type":"ContainerStarted","Data":"93c8bcf8e74e4c613c74fdc4ce048b9dc1e4827eadcd395c82f0f639d94fcd99"}
Sep 09 12:20:49.089310 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:49.089273    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4kv6h" event={"ID":"5fa079e3-edaf-4746-93c9-fa89298bbb94","Type":"ContainerStarted","Data":"b6f0ef0fdb2dda155b81e0dfcd9ed299a9dfef6c58006ddf7ea0163bbe5c24d0"}
Sep 09 12:20:49.106289 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:49.106209    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4kv6h" podStartSLOduration=68.142975953 podStartE2EDuration="1m9.106190483s" totalImagesPullingTime="963.21453ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:49.105824563 +0000 UTC m=+68.802389714" watchObservedRunningTime="2026-09-09 12:20:49.106190483 +0000 UTC m=+68.802755629"
Sep 09 12:20:51.081059 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.081027    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84d6d7cf9f-96n79"]
Sep 09 12:20:51.096961 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.096468    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"]
Sep 09 12:20:51.109252 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.109228    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"
Sep 09 12:20:51.113136 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.112913    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 09 12:20:51.113136 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.112934    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 09 12:20:51.113528 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.113415    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 09 12:20:51.113704 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.113688    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 09 12:20:51.114147 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.114129    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"]
Sep 09 12:20:51.143280 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.143253    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5719a1a8-0cc7-48a5-870f-87825d4f381e-tmp\") pod \"klusterlet-addon-workmgr-57796c8bd9-td6cp\" (UID: \"5719a1a8-0cc7-48a5-870f-87825d4f381e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"
Sep 09 12:20:51.143390 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.143288    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5719a1a8-0cc7-48a5-870f-87825d4f381e-klusterlet-config\") pod \"klusterlet-addon-workmgr-57796c8bd9-td6cp\" (UID: \"5719a1a8-0cc7-48a5-870f-87825d4f381e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"
Sep 09 12:20:51.143390 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.143338    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6fwr\" (UniqueName: \"kubernetes.io/projected/5719a1a8-0cc7-48a5-870f-87825d4f381e-kube-api-access-b6fwr\") pod \"klusterlet-addon-workmgr-57796c8bd9-td6cp\" (UID: \"5719a1a8-0cc7-48a5-870f-87825d4f381e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"
Sep 09 12:20:51.203697 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.203674    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68f8668478-rg5qk"]
Sep 09 12:20:51.218542 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.218522    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.229576 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.228912    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68f8668478-rg5qk"]
Sep 09 12:20:51.244516 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.244496    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6fwr\" (UniqueName: \"kubernetes.io/projected/5719a1a8-0cc7-48a5-870f-87825d4f381e-kube-api-access-b6fwr\") pod \"klusterlet-addon-workmgr-57796c8bd9-td6cp\" (UID: \"5719a1a8-0cc7-48a5-870f-87825d4f381e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"
Sep 09 12:20:51.244634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.244532    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6f9h\" (UniqueName: \"kubernetes.io/projected/9cff25b0-468a-4261-bd8e-a5560517ce8e-kube-api-access-n6f9h\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.244634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.244559    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cff25b0-468a-4261-bd8e-a5560517ce8e-trusted-ca\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.244634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.244603    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9cff25b0-468a-4261-bd8e-a5560517ce8e-registry-certificates\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.244761 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.244653    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cff25b0-468a-4261-bd8e-a5560517ce8e-bound-sa-token\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.244761 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.244675    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5719a1a8-0cc7-48a5-870f-87825d4f381e-tmp\") pod \"klusterlet-addon-workmgr-57796c8bd9-td6cp\" (UID: \"5719a1a8-0cc7-48a5-870f-87825d4f381e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"
Sep 09 12:20:51.244761 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.244699    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5719a1a8-0cc7-48a5-870f-87825d4f381e-klusterlet-config\") pod \"klusterlet-addon-workmgr-57796c8bd9-td6cp\" (UID: \"5719a1a8-0cc7-48a5-870f-87825d4f381e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"
Sep 09 12:20:51.244761 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.244722    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9cff25b0-468a-4261-bd8e-a5560517ce8e-ca-trust-extracted\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.244761 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.244753    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cff25b0-468a-4261-bd8e-a5560517ce8e-registry-tls\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.244953 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.244778    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9cff25b0-468a-4261-bd8e-a5560517ce8e-installation-pull-secrets\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.244953 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.244808    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9cff25b0-468a-4261-bd8e-a5560517ce8e-image-registry-private-configuration\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.245037 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.245022    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5719a1a8-0cc7-48a5-870f-87825d4f381e-tmp\") pod \"klusterlet-addon-workmgr-57796c8bd9-td6cp\" (UID: \"5719a1a8-0cc7-48a5-870f-87825d4f381e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"
Sep 09 12:20:51.247547 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.247526    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5719a1a8-0cc7-48a5-870f-87825d4f381e-klusterlet-config\") pod \"klusterlet-addon-workmgr-57796c8bd9-td6cp\" (UID: \"5719a1a8-0cc7-48a5-870f-87825d4f381e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"
Sep 09 12:20:51.258578 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.258554    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6fwr\" (UniqueName: \"kubernetes.io/projected/5719a1a8-0cc7-48a5-870f-87825d4f381e-kube-api-access-b6fwr\") pod \"klusterlet-addon-workmgr-57796c8bd9-td6cp\" (UID: \"5719a1a8-0cc7-48a5-870f-87825d4f381e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"
Sep 09 12:20:51.345443 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.345367    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9cff25b0-468a-4261-bd8e-a5560517ce8e-image-registry-private-configuration\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.345443 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.345419    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6f9h\" (UniqueName: \"kubernetes.io/projected/9cff25b0-468a-4261-bd8e-a5560517ce8e-kube-api-access-n6f9h\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.345668 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.345565    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cff25b0-468a-4261-bd8e-a5560517ce8e-trusted-ca\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.345668 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.345610    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9cff25b0-468a-4261-bd8e-a5560517ce8e-registry-certificates\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.345668 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.345653    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cff25b0-468a-4261-bd8e-a5560517ce8e-bound-sa-token\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.345780 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.345682    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9cff25b0-468a-4261-bd8e-a5560517ce8e-ca-trust-extracted\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.345780 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.345709    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cff25b0-468a-4261-bd8e-a5560517ce8e-registry-tls\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.345780 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.345732    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9cff25b0-468a-4261-bd8e-a5560517ce8e-installation-pull-secrets\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.346538 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.346510    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9cff25b0-468a-4261-bd8e-a5560517ce8e-registry-certificates\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.346641 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.346582    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cff25b0-468a-4261-bd8e-a5560517ce8e-trusted-ca\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.346641 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.346622    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9cff25b0-468a-4261-bd8e-a5560517ce8e-ca-trust-extracted\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.348468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.348410    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9cff25b0-468a-4261-bd8e-a5560517ce8e-image-registry-private-configuration\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.348468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.348424    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cff25b0-468a-4261-bd8e-a5560517ce8e-registry-tls\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.348754 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.348735    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9cff25b0-468a-4261-bd8e-a5560517ce8e-installation-pull-secrets\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.353745 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.353724    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cff25b0-468a-4261-bd8e-a5560517ce8e-bound-sa-token\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.354007 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.353990    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6f9h\" (UniqueName: \"kubernetes.io/projected/9cff25b0-468a-4261-bd8e-a5560517ce8e-kube-api-access-n6f9h\") pod \"image-registry-68f8668478-rg5qk\" (UID: \"9cff25b0-468a-4261-bd8e-a5560517ce8e\") " pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:51.420386 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.420349    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"
Sep 09 12:20:51.529179 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:51.529147    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:52.596913 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:52.596823    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"]
Sep 09 12:20:52.600797 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:20:52.600766    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5719a1a8_0cc7_48a5_870f_87825d4f381e.slice/crio-a31db56d63acf1a2316965e274a521237aa6d030d7aac11bef9ab319e3698bb4 WatchSource:0}: Error finding container a31db56d63acf1a2316965e274a521237aa6d030d7aac11bef9ab319e3698bb4: Status 404 returned error can't find the container with id a31db56d63acf1a2316965e274a521237aa6d030d7aac11bef9ab319e3698bb4
Sep 09 12:20:52.605491 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:52.605469    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68f8668478-rg5qk"]
Sep 09 12:20:52.608336 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:20:52.608314    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cff25b0_468a_4261_bd8e_a5560517ce8e.slice/crio-923e5b04c6220ef2251a4f1ea968177f6f2b8045403a80cdb24e8c4dfa81463f WatchSource:0}: Error finding container 923e5b04c6220ef2251a4f1ea968177f6f2b8045403a80cdb24e8c4dfa81463f: Status 404 returned error can't find the container with id 923e5b04c6220ef2251a4f1ea968177f6f2b8045403a80cdb24e8c4dfa81463f
Sep 09 12:20:53.103991 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:53.103952    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68f8668478-rg5qk" event={"ID":"9cff25b0-468a-4261-bd8e-a5560517ce8e","Type":"ContainerStarted","Data":"5b5a008e1f46776641f041ca1c3f28e51d87506d10ce17a0e82e8a233364c3c0"}
Sep 09 12:20:53.104175 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:53.104018    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68f8668478-rg5qk" event={"ID":"9cff25b0-468a-4261-bd8e-a5560517ce8e","Type":"ContainerStarted","Data":"923e5b04c6220ef2251a4f1ea968177f6f2b8045403a80cdb24e8c4dfa81463f"}
Sep 09 12:20:53.104175 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:53.104080    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:20:53.105202 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:53.105174    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp" event={"ID":"5719a1a8-0cc7-48a5-870f-87825d4f381e","Type":"ContainerStarted","Data":"a31db56d63acf1a2316965e274a521237aa6d030d7aac11bef9ab319e3698bb4"}
Sep 09 12:20:53.106607 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:53.106586    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b9h5d" event={"ID":"e2d11583-88d9-42e6-8de7-48df91ba8254","Type":"ContainerStarted","Data":"be3a8bbe4ae4cc13c3cadab0332909c094e77ef50bc2f3899de2081ceaa1557f"}
Sep 09 12:20:53.106737 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:53.106707    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:20:53.108488 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:53.108466    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lq49k" event={"ID":"72bee612-067e-4654-9095-41e974f65516","Type":"ContainerStarted","Data":"8e5d5643a4aa7ba75b6472a2eea31efde4ee43f57fa239f3bdcd686156095b25"}
Sep 09 12:20:53.108580 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:53.108510    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lq49k" event={"ID":"72bee612-067e-4654-9095-41e974f65516","Type":"ContainerStarted","Data":"437c3fe2b90f8eea6db184d798b494408cb582a79a6b80b69b09bdc612872855"}
Sep 09 12:20:53.108580 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:53.108536    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lq49k"
Sep 09 12:20:53.109924 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:53.109908    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lxbw5" event={"ID":"cb9acdab-04f6-47b1-9bcd-d24246dcf52a","Type":"ContainerStarted","Data":"353c9f765f349c5ab5ec2dbddcbd965e9e90a4d6c48722316fa16a2285f8592d"}
Sep 09 12:20:53.123728 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:53.123691    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68f8668478-rg5qk" podStartSLOduration=2.123680924 podStartE2EDuration="2.123680924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:53.122655454 +0000 UTC m=+72.819220603" watchObservedRunningTime="2026-09-09 12:20:53.123680924 +0000 UTC m=+72.820246066"
Sep 09 12:20:53.136753 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:53.136711    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-b9h5d" podStartSLOduration=67.754669348 podStartE2EDuration="1m13.13670206s" totalImagesPullingTime="5.382032712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:53.136378272 +0000 UTC m=+72.832943421" watchObservedRunningTime="2026-09-09 12:20:53.13670206 +0000 UTC m=+72.833267210"
Sep 09 12:20:53.152117 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:53.152076    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lq49k" podStartSLOduration=33.912609347 podStartE2EDuration="39.152064738s" totalImagesPullingTime="5.239455391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:53.151188871 +0000 UTC m=+72.847754022" watchObservedRunningTime="2026-09-09 12:20:53.152064738 +0000 UTC m=+72.848629889"
Sep 09 12:20:55.875334 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:55.875169    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lxbw5" podStartSLOduration=69.469835513 podStartE2EDuration="1m14.87514673s" totalImagesPullingTime="5.405311217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:53.164989764 +0000 UTC m=+72.861554915" watchObservedRunningTime="2026-09-09 12:20:55.87514673 +0000 UTC m=+75.571711881"
Sep 09 12:20:55.876326 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:55.876304    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"]
Sep 09 12:20:55.879652 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:55.879632    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"
Sep 09 12:20:55.882007 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:55.881981    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 12:20:55.882986 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:55.882812    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-t97r8\""
Sep 09 12:20:55.882986 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:55.882825    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 09 12:20:55.882986 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:55.882812    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 12:20:55.882986 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:55.882866    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 12:20:55.882986 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:55.882868    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 09 12:20:55.889380 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:55.889359    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"]
Sep 09 12:20:55.980688 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:55.980655    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbvzr\" (UniqueName: \"kubernetes.io/projected/47fae105-1e04-4d37-90ed-da1fa03d55b6-kube-api-access-zbvzr\") pod \"prometheus-operator-6f4946db56-pgmhg\" (UID: \"47fae105-1e04-4d37-90ed-da1fa03d55b6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"
Sep 09 12:20:55.980688 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:55.980692    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47fae105-1e04-4d37-90ed-da1fa03d55b6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pgmhg\" (UID: \"47fae105-1e04-4d37-90ed-da1fa03d55b6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"
Sep 09 12:20:55.980997 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:55.980727    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/47fae105-1e04-4d37-90ed-da1fa03d55b6-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pgmhg\" (UID: \"47fae105-1e04-4d37-90ed-da1fa03d55b6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"
Sep 09 12:20:55.980997 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:55.980831    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47fae105-1e04-4d37-90ed-da1fa03d55b6-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pgmhg\" (UID: \"47fae105-1e04-4d37-90ed-da1fa03d55b6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"
Sep 09 12:20:56.081577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:56.081532    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47fae105-1e04-4d37-90ed-da1fa03d55b6-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pgmhg\" (UID: \"47fae105-1e04-4d37-90ed-da1fa03d55b6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"
Sep 09 12:20:56.081760 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:56.081604    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbvzr\" (UniqueName: \"kubernetes.io/projected/47fae105-1e04-4d37-90ed-da1fa03d55b6-kube-api-access-zbvzr\") pod \"prometheus-operator-6f4946db56-pgmhg\" (UID: \"47fae105-1e04-4d37-90ed-da1fa03d55b6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"
Sep 09 12:20:56.081760 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:56.081635    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47fae105-1e04-4d37-90ed-da1fa03d55b6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pgmhg\" (UID: \"47fae105-1e04-4d37-90ed-da1fa03d55b6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"
Sep 09 12:20:56.081760 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:56.081663    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/47fae105-1e04-4d37-90ed-da1fa03d55b6-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pgmhg\" (UID: \"47fae105-1e04-4d37-90ed-da1fa03d55b6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"
Sep 09 12:20:56.082368 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:56.082342    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47fae105-1e04-4d37-90ed-da1fa03d55b6-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pgmhg\" (UID: \"47fae105-1e04-4d37-90ed-da1fa03d55b6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"
Sep 09 12:20:56.084332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:56.084288    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47fae105-1e04-4d37-90ed-da1fa03d55b6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pgmhg\" (UID: \"47fae105-1e04-4d37-90ed-da1fa03d55b6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"
Sep 09 12:20:56.084463 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:56.084445    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/47fae105-1e04-4d37-90ed-da1fa03d55b6-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pgmhg\" (UID: \"47fae105-1e04-4d37-90ed-da1fa03d55b6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"
Sep 09 12:20:56.089586 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:56.089566    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbvzr\" (UniqueName: \"kubernetes.io/projected/47fae105-1e04-4d37-90ed-da1fa03d55b6-kube-api-access-zbvzr\") pod \"prometheus-operator-6f4946db56-pgmhg\" (UID: \"47fae105-1e04-4d37-90ed-da1fa03d55b6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"
Sep 09 12:20:56.190586 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:56.190515    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"
Sep 09 12:20:57.127271 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:57.127241    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp" event={"ID":"5719a1a8-0cc7-48a5-870f-87825d4f381e","Type":"ContainerStarted","Data":"3b1240f8f696eb682d56ab8fcd894587d4b6f47e1515fca9659efafd01be2aea"}
Sep 09 12:20:57.127750 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:57.127428    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"
Sep 09 12:20:57.128651 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:57.128633    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp"
Sep 09 12:20:57.144368 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:57.144319    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57796c8bd9-td6cp" podStartSLOduration=1.771915219 podStartE2EDuration="6.144306686s" totalImagesPullingTime="4.372391467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:57.142920548 +0000 UTC m=+76.839485698" watchObservedRunningTime="2026-09-09 12:20:57.144306686 +0000 UTC m=+76.840871863"
Sep 09 12:20:57.169993 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:57.169940    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pgmhg"]
Sep 09 12:20:57.173072 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:20:57.173046    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47fae105_1e04_4d37_90ed_da1fa03d55b6.slice/crio-01461b795fd21bf299fd1e3e01fafc995fa721adc32b5bcebbc76813fb3bfba9 WatchSource:0}: Error finding container 01461b795fd21bf299fd1e3e01fafc995fa721adc32b5bcebbc76813fb3bfba9: Status 404 returned error can't find the container with id 01461b795fd21bf299fd1e3e01fafc995fa721adc32b5bcebbc76813fb3bfba9
Sep 09 12:20:58.131392 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:58.131358    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg" event={"ID":"47fae105-1e04-4d37-90ed-da1fa03d55b6","Type":"ContainerStarted","Data":"01461b795fd21bf299fd1e3e01fafc995fa721adc32b5bcebbc76813fb3bfba9"}
Sep 09 12:20:58.965984 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:58.965914    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-w66hv"]
Sep 09 12:20:58.969011 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:58.968982    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-w66hv"
Sep 09 12:20:58.971408 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:58.971391    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-rbl8r\""
Sep 09 12:20:58.971533 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:58.971401    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 09 12:20:58.971533 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:58.971497    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 09 12:20:58.977963 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:58.977943    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-w66hv"]
Sep 09 12:20:59.002681 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:59.002656    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fspvr\" (UniqueName: \"kubernetes.io/projected/1637635d-cc6c-490f-801d-f37a0b675865-kube-api-access-fspvr\") pod \"downloads-85d98c998f-w66hv\" (UID: \"1637635d-cc6c-490f-801d-f37a0b675865\") " pod="openshift-console/downloads-85d98c998f-w66hv"
Sep 09 12:20:59.103822 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:59.103802    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fspvr\" (UniqueName: \"kubernetes.io/projected/1637635d-cc6c-490f-801d-f37a0b675865-kube-api-access-fspvr\") pod \"downloads-85d98c998f-w66hv\" (UID: \"1637635d-cc6c-490f-801d-f37a0b675865\") " pod="openshift-console/downloads-85d98c998f-w66hv"
Sep 09 12:20:59.112466 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:59.112438    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fspvr\" (UniqueName: \"kubernetes.io/projected/1637635d-cc6c-490f-801d-f37a0b675865-kube-api-access-fspvr\") pod \"downloads-85d98c998f-w66hv\" (UID: \"1637635d-cc6c-490f-801d-f37a0b675865\") " pod="openshift-console/downloads-85d98c998f-w66hv"
Sep 09 12:20:59.138461 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:59.138435    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg" event={"ID":"47fae105-1e04-4d37-90ed-da1fa03d55b6","Type":"ContainerStarted","Data":"d197b3f3a68c5554a7c638fe8291be06eab17ebfd6c3facbc7ca5c94dafc72f9"}
Sep 09 12:20:59.138732 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:59.138489    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg" event={"ID":"47fae105-1e04-4d37-90ed-da1fa03d55b6","Type":"ContainerStarted","Data":"8825376cfa71bb4bacb4d0fca504a9913aa10956729dd105d1d4bafca59e1f14"}
Sep 09 12:20:59.154252 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:59.154215    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-pgmhg" podStartSLOduration=2.773214761 podStartE2EDuration="4.154202168s" totalImagesPullingTime="1.380987407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:59.153649989 +0000 UTC m=+78.850215134" watchObservedRunningTime="2026-09-09 12:20:59.154202168 +0000 UTC m=+78.850767319"
Sep 09 12:20:59.277806 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:59.277769    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-w66hv"
Sep 09 12:20:59.545119 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:20:59.545055    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-w66hv"]
Sep 09 12:20:59.548211 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:20:59.548187    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1637635d_cc6c_490f_801d_f37a0b675865.slice/crio-ae5270da9a9d77892629dd29aa637cf00cd059469bffc58bd15e1003ad8c384d WatchSource:0}: Error finding container ae5270da9a9d77892629dd29aa637cf00cd059469bffc58bd15e1003ad8c384d: Status 404 returned error can't find the container with id ae5270da9a9d77892629dd29aa637cf00cd059469bffc58bd15e1003ad8c384d
Sep 09 12:21:00.146661 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:00.146627    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-w66hv" event={"ID":"1637635d-cc6c-490f-801d-f37a0b675865","Type":"ContainerStarted","Data":"ae5270da9a9d77892629dd29aa637cf00cd059469bffc58bd15e1003ad8c384d"}
Sep 09 12:21:01.087525 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.087473    2582 patch_prober.go:28] interesting pod/image-registry-84d6d7cf9f-96n79 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 09 12:21:01.087692 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.087557    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79" podUID="68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 12:21:01.263615 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.263583    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"]
Sep 09 12:21:01.266568 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.266547    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"
Sep 09 12:21:01.269201 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.268928    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 09 12:21:01.269201 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.269028    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 09 12:21:01.269201 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.268933    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-jdmpj\""
Sep 09 12:21:01.272296 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.272276    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"]
Sep 09 12:21:01.275641 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.275615    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.277668 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.277646    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 09 12:21:01.277783 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.277761    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-thv6z\""
Sep 09 12:21:01.277905 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.277885    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 09 12:21:01.278375 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.278359    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 09 12:21:01.278511 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.278492    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"]
Sep 09 12:21:01.290312 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.290290    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ggzw9"]
Sep 09 12:21:01.293267 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.293249    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.295272 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.295252    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 12:21:01.295842 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.295823    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 12:21:01.295952 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.295876    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 12:21:01.296010 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.295828    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tnpqb\""
Sep 09 12:21:01.296161 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.296146    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"]
Sep 09 12:21:01.321937 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.321909    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36f7cc6e-6e5b-472f-b3f9-e1978d880277-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.322058 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.321955    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m5gw\" (UniqueName: \"kubernetes.io/projected/36f7cc6e-6e5b-472f-b3f9-e1978d880277-kube-api-access-8m5gw\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.322058 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.321989    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/36f7cc6e-6e5b-472f-b3f9-e1978d880277-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.322058 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.322050    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/92925e47-7972-411e-b1c4-7265ff222b6a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-w87hx\" (UID: \"92925e47-7972-411e-b1c4-7265ff222b6a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"
Sep 09 12:21:01.322226 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.322076    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92925e47-7972-411e-b1c4-7265ff222b6a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-w87hx\" (UID: \"92925e47-7972-411e-b1c4-7265ff222b6a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"
Sep 09 12:21:01.322226 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.322104    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/36f7cc6e-6e5b-472f-b3f9-e1978d880277-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.322226 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.322130    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pjlv\" (UniqueName: \"kubernetes.io/projected/92925e47-7972-411e-b1c4-7265ff222b6a-kube-api-access-9pjlv\") pod \"openshift-state-metrics-657fd6c54-w87hx\" (UID: \"92925e47-7972-411e-b1c4-7265ff222b6a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"
Sep 09 12:21:01.322226 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.322203    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/36f7cc6e-6e5b-472f-b3f9-e1978d880277-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.322415 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.322244    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92925e47-7972-411e-b1c4-7265ff222b6a-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-w87hx\" (UID: \"92925e47-7972-411e-b1c4-7265ff222b6a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"
Sep 09 12:21:01.322415 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.322274    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/36f7cc6e-6e5b-472f-b3f9-e1978d880277-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.422903 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.422811    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c89c9721-240b-4370-90fd-bf1484cef7a6-root\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.422903 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.422882    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92925e47-7972-411e-b1c4-7265ff222b6a-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-w87hx\" (UID: \"92925e47-7972-411e-b1c4-7265ff222b6a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"
Sep 09 12:21:01.423100 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.422908    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/36f7cc6e-6e5b-472f-b3f9-e1978d880277-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.423100 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.422933    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-tls\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.423100 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.422955    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-textfile\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.423100 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423076    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-wtmp\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.423287 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423105    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-accelerators-collector-config\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.423287 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423137    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36f7cc6e-6e5b-472f-b3f9-e1978d880277-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.423287 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423165    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m5gw\" (UniqueName: \"kubernetes.io/projected/36f7cc6e-6e5b-472f-b3f9-e1978d880277-kube-api-access-8m5gw\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.423287 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423199    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/36f7cc6e-6e5b-472f-b3f9-e1978d880277-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.423287 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423222    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c89c9721-240b-4370-90fd-bf1484cef7a6-sys\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.423287 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423249    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.423287 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423284    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/92925e47-7972-411e-b1c4-7265ff222b6a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-w87hx\" (UID: \"92925e47-7972-411e-b1c4-7265ff222b6a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"
Sep 09 12:21:01.423544 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423298    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/36f7cc6e-6e5b-472f-b3f9-e1978d880277-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.423544 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423305    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92925e47-7972-411e-b1c4-7265ff222b6a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-w87hx\" (UID: \"92925e47-7972-411e-b1c4-7265ff222b6a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"
Sep 09 12:21:01.423544 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423365    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/36f7cc6e-6e5b-472f-b3f9-e1978d880277-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.423544 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423397    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pjlv\" (UniqueName: \"kubernetes.io/projected/92925e47-7972-411e-b1c4-7265ff222b6a-kube-api-access-9pjlv\") pod \"openshift-state-metrics-657fd6c54-w87hx\" (UID: \"92925e47-7972-411e-b1c4-7265ff222b6a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"
Sep 09 12:21:01.423544 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423431    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9qwm\" (UniqueName: \"kubernetes.io/projected/c89c9721-240b-4370-90fd-bf1484cef7a6-kube-api-access-m9qwm\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.423544 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423463    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/36f7cc6e-6e5b-472f-b3f9-e1978d880277-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.423544 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423500    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c89c9721-240b-4370-90fd-bf1484cef7a6-metrics-client-ca\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.423915 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.423551    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92925e47-7972-411e-b1c4-7265ff222b6a-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-w87hx\" (UID: \"92925e47-7972-411e-b1c4-7265ff222b6a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"
Sep 09 12:21:01.423915 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:21:01.423677    2582 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 09 12:21:01.423915 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:21:01.423732    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36f7cc6e-6e5b-472f-b3f9-e1978d880277-kube-state-metrics-tls podName:36f7cc6e-6e5b-472f-b3f9-e1978d880277 nodeName:}" failed. No retries permitted until 2026-09-09 12:21:01.923712935 +0000 UTC m=+81.620278069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/36f7cc6e-6e5b-472f-b3f9-e1978d880277-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-rqwnc" (UID: "36f7cc6e-6e5b-472f-b3f9-e1978d880277") : secret "kube-state-metrics-tls" not found
Sep 09 12:21:01.424375 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.424349    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/36f7cc6e-6e5b-472f-b3f9-e1978d880277-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.424375 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.424370    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36f7cc6e-6e5b-472f-b3f9-e1978d880277-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.426602 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.426580    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/92925e47-7972-411e-b1c4-7265ff222b6a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-w87hx\" (UID: \"92925e47-7972-411e-b1c4-7265ff222b6a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"
Sep 09 12:21:01.427025 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.427006    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/36f7cc6e-6e5b-472f-b3f9-e1978d880277-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.427463 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.427443    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92925e47-7972-411e-b1c4-7265ff222b6a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-w87hx\" (UID: \"92925e47-7972-411e-b1c4-7265ff222b6a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"
Sep 09 12:21:01.434660 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.434637    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m5gw\" (UniqueName: \"kubernetes.io/projected/36f7cc6e-6e5b-472f-b3f9-e1978d880277-kube-api-access-8m5gw\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.434794 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.434761    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pjlv\" (UniqueName: \"kubernetes.io/projected/92925e47-7972-411e-b1c4-7265ff222b6a-kube-api-access-9pjlv\") pod \"openshift-state-metrics-657fd6c54-w87hx\" (UID: \"92925e47-7972-411e-b1c4-7265ff222b6a\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"
Sep 09 12:21:01.524097 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.524057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9qwm\" (UniqueName: \"kubernetes.io/projected/c89c9721-240b-4370-90fd-bf1484cef7a6-kube-api-access-m9qwm\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.524260 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.524109    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c89c9721-240b-4370-90fd-bf1484cef7a6-metrics-client-ca\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.524260 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.524142    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c89c9721-240b-4370-90fd-bf1484cef7a6-root\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.524260 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.524185    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-tls\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.524260 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.524211    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-textfile\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.524260 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.524234    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-wtmp\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.524260 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.524250    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-accelerators-collector-config\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.524555 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.524264    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c89c9721-240b-4370-90fd-bf1484cef7a6-root\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.524555 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.524325    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c89c9721-240b-4370-90fd-bf1484cef7a6-sys\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.524555 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.524346    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.524555 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:21:01.524369    2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 09 12:21:01.524555 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:21:01.524424    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-tls podName:c89c9721-240b-4370-90fd-bf1484cef7a6 nodeName:}" failed. No retries permitted until 2026-09-09 12:21:02.024405791 +0000 UTC m=+81.720970926 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-tls") pod "node-exporter-ggzw9" (UID: "c89c9721-240b-4370-90fd-bf1484cef7a6") : secret "node-exporter-tls" not found
Sep 09 12:21:01.524802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.524563    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-wtmp\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.524802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.524608    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c89c9721-240b-4370-90fd-bf1484cef7a6-sys\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.524802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.524785    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c89c9721-240b-4370-90fd-bf1484cef7a6-metrics-client-ca\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.524944 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.524815    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-accelerators-collector-config\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.524987 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.524970    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-textfile\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.527290 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.527262    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.533315 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.533295    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9qwm\" (UniqueName: \"kubernetes.io/projected/c89c9721-240b-4370-90fd-bf1484cef7a6-kube-api-access-m9qwm\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:01.577920 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.577897    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"
Sep 09 12:21:01.856970 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.854255    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx"]
Sep 09 12:21:01.859945 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:21:01.859919    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92925e47_7972_411e_b1c4_7265ff222b6a.slice/crio-1261630da400d060d829c5fa37914d7d10f710b7b7fd25ccb23bcbc672863c0c WatchSource:0}: Error finding container 1261630da400d060d829c5fa37914d7d10f710b7b7fd25ccb23bcbc672863c0c: Status 404 returned error can't find the container with id 1261630da400d060d829c5fa37914d7d10f710b7b7fd25ccb23bcbc672863c0c
Sep 09 12:21:01.929027 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.928993    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/36f7cc6e-6e5b-472f-b3f9-e1978d880277-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:01.931905 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:01.931825    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/36f7cc6e-6e5b-472f-b3f9-e1978d880277-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-rqwnc\" (UID: \"36f7cc6e-6e5b-472f-b3f9-e1978d880277\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:02.030302 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:02.030269    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-tls\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:02.032642 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:02.032613    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c89c9721-240b-4370-90fd-bf1484cef7a6-node-exporter-tls\") pod \"node-exporter-ggzw9\" (UID: \"c89c9721-240b-4370-90fd-bf1484cef7a6\") " pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:02.157727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:02.157583    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx" event={"ID":"92925e47-7972-411e-b1c4-7265ff222b6a","Type":"ContainerStarted","Data":"5db720c209528afbd08878b54cf442a485f9c0e82a0cc58ebcc448103f1c7906"}
Sep 09 12:21:02.157727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:02.157649    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx" event={"ID":"92925e47-7972-411e-b1c4-7265ff222b6a","Type":"ContainerStarted","Data":"e05f6d5850d039eec24159c8860dba64c6cb51015d3830999e821aeff3bdb0b8"}
Sep 09 12:21:02.157727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:02.157679    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx" event={"ID":"92925e47-7972-411e-b1c4-7265ff222b6a","Type":"ContainerStarted","Data":"1261630da400d060d829c5fa37914d7d10f710b7b7fd25ccb23bcbc672863c0c"}
Sep 09 12:21:02.187771 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:02.187747    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"
Sep 09 12:21:02.204696 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:02.203446    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ggzw9"
Sep 09 12:21:02.213688 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:21:02.213664    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc89c9721_240b_4370_90fd_bf1484cef7a6.slice/crio-f32ed62c678c56e8d60212fb7b6ff2b90a4252b2cb5594c3e7b95ffed7986069 WatchSource:0}: Error finding container f32ed62c678c56e8d60212fb7b6ff2b90a4252b2cb5594c3e7b95ffed7986069: Status 404 returned error can't find the container with id f32ed62c678c56e8d60212fb7b6ff2b90a4252b2cb5594c3e7b95ffed7986069
Sep 09 12:21:02.466635 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:02.466567    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc"]
Sep 09 12:21:02.470088 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:21:02.470059    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f7cc6e_6e5b_472f_b3f9_e1978d880277.slice/crio-4ef6d32ad306086cb264e4a6b72d2176655b4d169abfdee938419dca06548b8b WatchSource:0}: Error finding container 4ef6d32ad306086cb264e4a6b72d2176655b4d169abfdee938419dca06548b8b: Status 404 returned error can't find the container with id 4ef6d32ad306086cb264e4a6b72d2176655b4d169abfdee938419dca06548b8b
Sep 09 12:21:03.117219 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:03.117173    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lq49k"
Sep 09 12:21:03.166638 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:03.166404    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc" event={"ID":"36f7cc6e-6e5b-472f-b3f9-e1978d880277","Type":"ContainerStarted","Data":"4ef6d32ad306086cb264e4a6b72d2176655b4d169abfdee938419dca06548b8b"}
Sep 09 12:21:03.180165 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:03.180088    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ggzw9" event={"ID":"c89c9721-240b-4370-90fd-bf1484cef7a6","Type":"ContainerStarted","Data":"f32ed62c678c56e8d60212fb7b6ff2b90a4252b2cb5594c3e7b95ffed7986069"}
Sep 09 12:21:04.187226 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:04.187198    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx" event={"ID":"92925e47-7972-411e-b1c4-7265ff222b6a","Type":"ContainerStarted","Data":"2acfd9225bcf430efe69b402cddde2a3b0d7f03a3f49c58c1e25612117b3512f"}
Sep 09 12:21:04.189206 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:04.189177    2582 generic.go:356] "Generic (PLEG): container finished" podID="c89c9721-240b-4370-90fd-bf1484cef7a6" containerID="f7034e8b9925cd9f677ec71d1313957f7991d556f8c2ffcebf97189b8777c06c" exitCode=0
Sep 09 12:21:04.189309 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:04.189242    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ggzw9" event={"ID":"c89c9721-240b-4370-90fd-bf1484cef7a6","Type":"ContainerDied","Data":"f7034e8b9925cd9f677ec71d1313957f7991d556f8c2ffcebf97189b8777c06c"}
Sep 09 12:21:04.206983 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:04.206931    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-w87hx" podStartSLOduration=2.104875717 podStartE2EDuration="3.206911342s" totalImagesPullingTime="1.102035625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:21:04.20519643 +0000 UTC m=+83.901761594" watchObservedRunningTime="2026-09-09 12:21:04.206911342 +0000 UTC m=+83.903476495"
Sep 09 12:21:05.196145 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:05.196035    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ggzw9" event={"ID":"c89c9721-240b-4370-90fd-bf1484cef7a6","Type":"ContainerStarted","Data":"5d0b15b6f99de519a4e956b3eab336d546624c53397a11893cb1e3030d03f92e"}
Sep 09 12:21:05.196145 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:05.196107    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ggzw9" event={"ID":"c89c9721-240b-4370-90fd-bf1484cef7a6","Type":"ContainerStarted","Data":"9e2aeded920c424195b6563a8f3696d0dae6888225b62f46dd684d9545df2574"}
Sep 09 12:21:05.201397 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:05.201355    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc" event={"ID":"36f7cc6e-6e5b-472f-b3f9-e1978d880277","Type":"ContainerStarted","Data":"c522b2720a9514255a83dd4586a52c610d6ed45931a1da70297d6eae848217dd"}
Sep 09 12:21:05.201545 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:05.201410    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc" event={"ID":"36f7cc6e-6e5b-472f-b3f9-e1978d880277","Type":"ContainerStarted","Data":"f233bb1f99e984d79600c246fd0d5dc3e03755b74c6901282cf3b854c929c5fa"}
Sep 09 12:21:05.201545 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:05.201432    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc" event={"ID":"36f7cc6e-6e5b-472f-b3f9-e1978d880277","Type":"ContainerStarted","Data":"d6b4c2827772286f0df45b2ea7415f6a7cadd3cbd1f94d8a0a9f67166ccc4dcd"}
Sep 09 12:21:05.219388 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:05.219347    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ggzw9" podStartSLOduration=2.816225987 podStartE2EDuration="4.219335327s" totalImagesPullingTime="1.323587194s" totalInitContainerRuntime="79.522146ms" isStatefulPod=true podCreationTimestamp="2026-09-09 12:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:21:05.218676149 +0000 UTC m=+84.915241294" watchObservedRunningTime="2026-09-09 12:21:05.219335327 +0000 UTC m=+84.915900474"
Sep 09 12:21:05.238330 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:05.238287    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-rqwnc" podStartSLOduration=2.564700245 podStartE2EDuration="4.238276664s" totalImagesPullingTime="1.673576419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:21:05.237120784 +0000 UTC m=+84.933685954" watchObservedRunningTime="2026-09-09 12:21:05.238276664 +0000 UTC m=+84.934841814"
Sep 09 12:21:06.011833 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:06.011779    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx"]
Sep 09 12:21:06.015331 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:06.015310    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx"
Sep 09 12:21:06.017118 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:06.017085    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 09 12:21:06.017252 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:06.017159    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-4lnx8\""
Sep 09 12:21:06.022510 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:06.022485    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx"]
Sep 09 12:21:06.065090 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:06.065064    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/704f3e3e-97a3-41b4-bd6d-c148e147ad40-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-r5ggx\" (UID: \"704f3e3e-97a3-41b4-bd6d-c148e147ad40\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx"
Sep 09 12:21:06.065222 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:06.065105    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg9s8\" (UniqueName: \"kubernetes.io/projected/704f3e3e-97a3-41b4-bd6d-c148e147ad40-kube-api-access-jg9s8\") pod \"monitoring-plugin-85bbbbbf4-r5ggx\" (UID: \"704f3e3e-97a3-41b4-bd6d-c148e147ad40\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx"
Sep 09 12:21:06.166497 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:06.166467    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/704f3e3e-97a3-41b4-bd6d-c148e147ad40-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-r5ggx\" (UID: \"704f3e3e-97a3-41b4-bd6d-c148e147ad40\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx"
Sep 09 12:21:06.166497 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:06.166497    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg9s8\" (UniqueName: \"kubernetes.io/projected/704f3e3e-97a3-41b4-bd6d-c148e147ad40-kube-api-access-jg9s8\") pod \"monitoring-plugin-85bbbbbf4-r5ggx\" (UID: \"704f3e3e-97a3-41b4-bd6d-c148e147ad40\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx"
Sep 09 12:21:06.166686 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:21:06.166601    2582 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 09 12:21:06.166686 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:21:06.166659    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/704f3e3e-97a3-41b4-bd6d-c148e147ad40-monitoring-plugin-cert podName:704f3e3e-97a3-41b4-bd6d-c148e147ad40 nodeName:}" failed. No retries permitted until 2026-09-09 12:21:06.666643972 +0000 UTC m=+86.363209102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/704f3e3e-97a3-41b4-bd6d-c148e147ad40-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-r5ggx" (UID: "704f3e3e-97a3-41b4-bd6d-c148e147ad40") : secret "monitoring-plugin-cert" not found
Sep 09 12:21:06.175981 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:06.175954    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg9s8\" (UniqueName: \"kubernetes.io/projected/704f3e3e-97a3-41b4-bd6d-c148e147ad40-kube-api-access-jg9s8\") pod \"monitoring-plugin-85bbbbbf4-r5ggx\" (UID: \"704f3e3e-97a3-41b4-bd6d-c148e147ad40\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx"
Sep 09 12:21:06.670620 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:06.670588    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/704f3e3e-97a3-41b4-bd6d-c148e147ad40-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-r5ggx\" (UID: \"704f3e3e-97a3-41b4-bd6d-c148e147ad40\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx"
Sep 09 12:21:06.673360 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:06.673332    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/704f3e3e-97a3-41b4-bd6d-c148e147ad40-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-r5ggx\" (UID: \"704f3e3e-97a3-41b4-bd6d-c148e147ad40\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx"
Sep 09 12:21:06.928191 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:06.928113    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx"
Sep 09 12:21:07.203824 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:07.203748    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx"]
Sep 09 12:21:07.207676 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:21:07.207652    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod704f3e3e_97a3_41b4_bd6d_c148e147ad40.slice/crio-fce3c09c35144950f4cff696e4637e2f0466d7bb3e7bd28baa67865542623251 WatchSource:0}: Error finding container fce3c09c35144950f4cff696e4637e2f0466d7bb3e7bd28baa67865542623251: Status 404 returned error can't find the container with id fce3c09c35144950f4cff696e4637e2f0466d7bb3e7bd28baa67865542623251
Sep 09 12:21:08.215736 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:08.215663    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx" event={"ID":"704f3e3e-97a3-41b4-bd6d-c148e147ad40","Type":"ContainerStarted","Data":"fce3c09c35144950f4cff696e4637e2f0466d7bb3e7bd28baa67865542623251"}
Sep 09 12:21:09.067222 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.067192    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67d4699c9-4bfpg"]
Sep 09 12:21:09.070335 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.070314    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.073544 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.073524    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 09 12:21:09.073663 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.073577    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 09 12:21:09.073663 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.073528    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vpxwc\""
Sep 09 12:21:09.073663 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.073579    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 09 12:21:09.073822 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.073581    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 09 12:21:09.073822 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.073528    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 09 12:21:09.078946 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.078928    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67d4699c9-4bfpg"]
Sep 09 12:21:09.191795 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.191764    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-serving-cert\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.191795 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.191795    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-oauth-serving-cert\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.192024 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.191821    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-oauth-config\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.192024 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.191861    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-service-ca\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.192024 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.191881    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-config\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.192024 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.191896    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w4s7\" (UniqueName: \"kubernetes.io/projected/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-kube-api-access-9w4s7\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.221668 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.221637    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx" event={"ID":"704f3e3e-97a3-41b4-bd6d-c148e147ad40","Type":"ContainerStarted","Data":"e46a463142c22eca6073eccc0537db0b1b0274506ae0de1a328bf5e5dc21615e"}
Sep 09 12:21:09.222103 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.221839    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx"
Sep 09 12:21:09.227795 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.227775    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx"
Sep 09 12:21:09.237703 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.237660    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-r5ggx" podStartSLOduration=2.776879976 podStartE2EDuration="4.237647589s" totalImagesPullingTime="1.460767613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:21:09.236492785 +0000 UTC m=+88.933057937" watchObservedRunningTime="2026-09-09 12:21:09.237647589 +0000 UTC m=+88.934212739"
Sep 09 12:21:09.292654 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.292622    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-serving-cert\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.292806 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.292666    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-oauth-serving-cert\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.292806 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.292712    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-oauth-config\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.292806 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.292742    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-service-ca\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.292806 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.292772    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-config\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.292806 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.292796    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w4s7\" (UniqueName: \"kubernetes.io/projected/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-kube-api-access-9w4s7\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.293614 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.293568    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-oauth-serving-cert\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.293720 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.293655    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-service-ca\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.293940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.293908    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-config\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.295464 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.295436    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-serving-cert\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.295796 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.295772    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-oauth-config\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.301421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.301400    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w4s7\" (UniqueName: \"kubernetes.io/projected/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-kube-api-access-9w4s7\") pod \"console-67d4699c9-4bfpg\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") " pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.381510 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.381439    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:09.666265 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:09.666182    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67d4699c9-4bfpg"]
Sep 09 12:21:09.669139 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:21:09.669081    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65f6bfd9_bb3c_47ed_8e53_55fa5884e064.slice/crio-980e2c6361fac6d8c4b52300c2bbf6b231e90a3aaaf51591bfe689319ff9e77b WatchSource:0}: Error finding container 980e2c6361fac6d8c4b52300c2bbf6b231e90a3aaaf51591bfe689319ff9e77b: Status 404 returned error can't find the container with id 980e2c6361fac6d8c4b52300c2bbf6b231e90a3aaaf51591bfe689319ff9e77b
Sep 09 12:21:10.227891 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:10.227830    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d4699c9-4bfpg" event={"ID":"65f6bfd9-bb3c-47ed-8e53-55fa5884e064","Type":"ContainerStarted","Data":"980e2c6361fac6d8c4b52300c2bbf6b231e90a3aaaf51591bfe689319ff9e77b"}
Sep 09 12:21:11.086240 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:11.086205    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:21:14.119257 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:14.119224    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-68f8668478-rg5qk"
Sep 09 12:21:16.102836 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:16.102794    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79" podUID="68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c" containerName="registry" containerID="cri-o://7b1efe2af7d027efa1bbccb6f001896872ed2d917c27da2b3caf945fa8397957" gracePeriod=30
Sep 09 12:21:17.072990 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.072962    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:21:17.078722 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.078699    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59c7f676cf-zq5pq"]
Sep 09 12:21:17.078998 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.078986    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c" containerName="registry"
Sep 09 12:21:17.079042 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.079001    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c" containerName="registry"
Sep 09 12:21:17.079077 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.079048    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c" containerName="registry"
Sep 09 12:21:17.101162 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.101139    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59c7f676cf-zq5pq"]
Sep 09 12:21:17.101280 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.101243    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.107513 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.107408    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 09 12:21:17.163316 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.163260    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-ca-trust-extracted\") pod \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") "
Sep 09 12:21:17.163316 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.163294    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-image-registry-private-configuration\") pod \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") "
Sep 09 12:21:17.163470 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.163336    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqjs8\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-kube-api-access-mqjs8\") pod \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") "
Sep 09 12:21:17.163470 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.163445    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-installation-pull-secrets\") pod \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") "
Sep 09 12:21:17.163579 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.163474    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-bound-sa-token\") pod \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") "
Sep 09 12:21:17.163579 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.163515    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls\") pod \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") "
Sep 09 12:21:17.163579 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.163544    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-certificates\") pod \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") "
Sep 09 12:21:17.163723 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.163578    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-trusted-ca\") pod \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\" (UID: \"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c\") "
Sep 09 12:21:17.163723 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.163706    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-console-config\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.163819 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.163750    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-oauth-serving-cert\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.163819 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.163788    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-trusted-ca-bundle\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.163954 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.163835    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6lsr\" (UniqueName: \"kubernetes.io/projected/4d511574-c055-44a4-bf8a-260265e399b8-kube-api-access-b6lsr\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.163954 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.163894    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d511574-c055-44a4-bf8a-260265e399b8-console-oauth-config\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.163954 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.163939    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-service-ca\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.164091 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.163998    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d511574-c055-44a4-bf8a-260265e399b8-console-serving-cert\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.164091 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.164038    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c" (UID: "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:21:17.164461 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.164439    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c" (UID: "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:21:17.166204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.166142    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c" (UID: "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:21:17.166302 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.166211    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c" (UID: "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:21:17.166484 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.166461    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-kube-api-access-mqjs8" (OuterVolumeSpecName: "kube-api-access-mqjs8") pod "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c" (UID: "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c"). InnerVolumeSpecName "kube-api-access-mqjs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:21:17.166546 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.166513    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c" (UID: "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:21:17.166663 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.166642    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c" (UID: "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:21:17.172492 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.172472    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c" (UID: "68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:21:17.263079 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.263052    2582 generic.go:356] "Generic (PLEG): container finished" podID="68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c" containerID="7b1efe2af7d027efa1bbccb6f001896872ed2d917c27da2b3caf945fa8397957" exitCode=0
Sep 09 12:21:17.263157 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.263104    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79" event={"ID":"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c","Type":"ContainerDied","Data":"7b1efe2af7d027efa1bbccb6f001896872ed2d917c27da2b3caf945fa8397957"}
Sep 09 12:21:17.263157 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.263117    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79"
Sep 09 12:21:17.263233 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.263162    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84d6d7cf9f-96n79" event={"ID":"68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c","Type":"ContainerDied","Data":"6e9878f18d816c2e1c62528600a70caba1262bccdba2d479c5f9441c29bfe990"}
Sep 09 12:21:17.263233 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.263182    2582 scope.go:117] "RemoveContainer" containerID="7b1efe2af7d027efa1bbccb6f001896872ed2d917c27da2b3caf945fa8397957"
Sep 09 12:21:17.264457 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.264291    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6lsr\" (UniqueName: \"kubernetes.io/projected/4d511574-c055-44a4-bf8a-260265e399b8-kube-api-access-b6lsr\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.264457 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.264319    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d511574-c055-44a4-bf8a-260265e399b8-console-oauth-config\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.264457 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.264345    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-service-ca\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.264457 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.264373    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d511574-c055-44a4-bf8a-260265e399b8-console-serving-cert\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.264457 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.264403    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-console-config\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.264457 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.264426    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-oauth-serving-cert\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.264457 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.264453    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-trusted-ca-bundle\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.264817 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.264499    2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:21:17.264817 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.264514    2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-registry-certificates\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:21:17.264817 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.264528    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-trusted-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:21:17.264817 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.264542    2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-ca-trust-extracted\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:21:17.264817 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.264558    2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-image-registry-private-configuration\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:21:17.264817 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.264572    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqjs8\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-kube-api-access-mqjs8\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:21:17.264817 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.264639    2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-installation-pull-secrets\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:21:17.264817 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.264652    2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c-bound-sa-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:21:17.265246 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.265223    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-console-config\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.265486 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.265263    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-service-ca\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.265660 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.265635    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-trusted-ca-bundle\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.266121 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.266091    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-oauth-serving-cert\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.267011 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.266986    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d511574-c055-44a4-bf8a-260265e399b8-console-oauth-config\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.267212 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.267192    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d511574-c055-44a4-bf8a-260265e399b8-console-serving-cert\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.271911 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.271886    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6lsr\" (UniqueName: \"kubernetes.io/projected/4d511574-c055-44a4-bf8a-260265e399b8-kube-api-access-b6lsr\") pod \"console-59c7f676cf-zq5pq\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") " pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.284720 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.284699    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84d6d7cf9f-96n79"]
Sep 09 12:21:17.289249 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.289229    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-84d6d7cf9f-96n79"]
Sep 09 12:21:17.303980 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.303965    2582 scope.go:117] "RemoveContainer" containerID="7b1efe2af7d027efa1bbccb6f001896872ed2d917c27da2b3caf945fa8397957"
Sep 09 12:21:17.304312 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:21:17.304284    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b1efe2af7d027efa1bbccb6f001896872ed2d917c27da2b3caf945fa8397957\": container with ID starting with 7b1efe2af7d027efa1bbccb6f001896872ed2d917c27da2b3caf945fa8397957 not found: ID does not exist" containerID="7b1efe2af7d027efa1bbccb6f001896872ed2d917c27da2b3caf945fa8397957"
Sep 09 12:21:17.304388 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.304328    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b1efe2af7d027efa1bbccb6f001896872ed2d917c27da2b3caf945fa8397957"} err="failed to get container status \"7b1efe2af7d027efa1bbccb6f001896872ed2d917c27da2b3caf945fa8397957\": rpc error: code = NotFound desc = could not find container \"7b1efe2af7d027efa1bbccb6f001896872ed2d917c27da2b3caf945fa8397957\": container with ID starting with 7b1efe2af7d027efa1bbccb6f001896872ed2d917c27da2b3caf945fa8397957 not found: ID does not exist"
Sep 09 12:21:17.410403 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.410377    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:17.694915 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:17.694522    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59c7f676cf-zq5pq"]
Sep 09 12:21:18.271729 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:18.271691    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c7f676cf-zq5pq" event={"ID":"4d511574-c055-44a4-bf8a-260265e399b8","Type":"ContainerStarted","Data":"25b719aa6d2370195e694a52920a863285a88db91fa56db5b7c41bc2161ac67a"}
Sep 09 12:21:18.274043 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:18.274005    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-w66hv" event={"ID":"1637635d-cc6c-490f-801d-f37a0b675865","Type":"ContainerStarted","Data":"78d3f64a645dd10b87dca6a1f028282eddf04a734f22827475278abb166fe934"}
Sep 09 12:21:18.274570 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:18.274311    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-w66hv"
Sep 09 12:21:18.286556 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:18.286502    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-w66hv"
Sep 09 12:21:18.298429 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:18.298310    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-w66hv" podStartSLOduration=2.496889402 podStartE2EDuration="20.298290678s" totalImagesPullingTime="17.801401276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:21:18.297661027 +0000 UTC m=+97.994226179" watchObservedRunningTime="2026-09-09 12:21:18.298290678 +0000 UTC m=+97.994855832"
Sep 09 12:21:18.825945 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:18.825907    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c" path="/var/lib/kubelet/pods/68e1f6c9-b2ee-43cc-ac2b-cc3fc01c846c/volumes"
Sep 09 12:21:22.297239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:22.297199    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c7f676cf-zq5pq" event={"ID":"4d511574-c055-44a4-bf8a-260265e399b8","Type":"ContainerStarted","Data":"66bacdcdd44133be767082926cfd7aca7adc8b0e5a1d3c9b006fd2de503b7f49"}
Sep 09 12:21:22.299236 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:22.299206    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d4699c9-4bfpg" event={"ID":"65f6bfd9-bb3c-47ed-8e53-55fa5884e064","Type":"ContainerStarted","Data":"388cf131bfb45e1e4ee31be5febbdf028f2bc915c425b77d1de9956b6f9594ff"}
Sep 09 12:21:22.316198 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:22.316139    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59c7f676cf-zq5pq" podStartSLOduration=1.301590328 podStartE2EDuration="5.316112696s" totalImagesPullingTime="4.014522368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:21:22.31583617 +0000 UTC m=+102.012401335" watchObservedRunningTime="2026-09-09 12:21:22.316112696 +0000 UTC m=+102.012677859"
Sep 09 12:21:22.339372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:22.339314    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67d4699c9-4bfpg" podStartSLOduration=1.426122473 podStartE2EDuration="13.339298372s" totalImagesPullingTime="11.913175899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:21:22.337199593 +0000 UTC m=+102.033764752" watchObservedRunningTime="2026-09-09 12:21:22.339298372 +0000 UTC m=+102.035863525"
Sep 09 12:21:24.118896 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:24.118843    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-b9h5d"
Sep 09 12:21:27.411231 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:27.411194    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:27.411645 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:27.411259    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:27.416333 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:27.416308    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:28.327686 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:28.327660    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:21:28.376944 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:28.376529    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67d4699c9-4bfpg"]
Sep 09 12:21:29.382514 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:29.382484    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:40.684403 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:40.684379    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z8n7s_9e6412b5-162d-4538-8533-8806bbe0e8c9/serve-healthcheck-canary/0.log"
Sep 09 12:21:53.401340 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.401274    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67d4699c9-4bfpg" podUID="65f6bfd9-bb3c-47ed-8e53-55fa5884e064" containerName="console" containerID="cri-o://388cf131bfb45e1e4ee31be5febbdf028f2bc915c425b77d1de9956b6f9594ff" gracePeriod=15
Sep 09 12:21:53.706388 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.706360    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:53.765265 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.765234    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w4s7\" (UniqueName: \"kubernetes.io/projected/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-kube-api-access-9w4s7\") pod \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") "
Sep 09 12:21:53.765432 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.765274    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-service-ca\") pod \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") "
Sep 09 12:21:53.765432 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.765345    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-serving-cert\") pod \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") "
Sep 09 12:21:53.765432 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.765375    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-oauth-serving-cert\") pod \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") "
Sep 09 12:21:53.765607 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.765457    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-config\") pod \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") "
Sep 09 12:21:53.765607 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.765484    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-oauth-config\") pod \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\" (UID: \"65f6bfd9-bb3c-47ed-8e53-55fa5884e064\") "
Sep 09 12:21:53.765767 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.765716    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-service-ca" (OuterVolumeSpecName: "service-ca") pod "65f6bfd9-bb3c-47ed-8e53-55fa5884e064" (UID: "65f6bfd9-bb3c-47ed-8e53-55fa5884e064"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:21:53.765767 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.765734    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "65f6bfd9-bb3c-47ed-8e53-55fa5884e064" (UID: "65f6bfd9-bb3c-47ed-8e53-55fa5884e064"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:21:53.765969 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.765825    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-config" (OuterVolumeSpecName: "console-config") pod "65f6bfd9-bb3c-47ed-8e53-55fa5884e064" (UID: "65f6bfd9-bb3c-47ed-8e53-55fa5884e064"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:21:53.767641 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.767608    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "65f6bfd9-bb3c-47ed-8e53-55fa5884e064" (UID: "65f6bfd9-bb3c-47ed-8e53-55fa5884e064"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:21:53.767767 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.767691    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-kube-api-access-9w4s7" (OuterVolumeSpecName: "kube-api-access-9w4s7") pod "65f6bfd9-bb3c-47ed-8e53-55fa5884e064" (UID: "65f6bfd9-bb3c-47ed-8e53-55fa5884e064"). InnerVolumeSpecName "kube-api-access-9w4s7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:21:53.767767 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.767683    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "65f6bfd9-bb3c-47ed-8e53-55fa5884e064" (UID: "65f6bfd9-bb3c-47ed-8e53-55fa5884e064"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:21:53.866786 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.866754    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-config\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:21:53.866786 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.866782    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-oauth-config\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:21:53.866786 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.866792    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w4s7\" (UniqueName: \"kubernetes.io/projected/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-kube-api-access-9w4s7\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:21:53.867019 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.866801    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:21:53.867019 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.866809    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-console-serving-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:21:53.867019 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:53.866816    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65f6bfd9-bb3c-47ed-8e53-55fa5884e064-oauth-serving-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:21:54.421902 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:54.421867    2582 generic.go:356] "Generic (PLEG): container finished" podID="65f6bfd9-bb3c-47ed-8e53-55fa5884e064" containerID="388cf131bfb45e1e4ee31be5febbdf028f2bc915c425b77d1de9956b6f9594ff" exitCode=0
Sep 09 12:21:54.422332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:54.421914    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d4699c9-4bfpg"
Sep 09 12:21:54.422332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:54.421943    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d4699c9-4bfpg" event={"ID":"65f6bfd9-bb3c-47ed-8e53-55fa5884e064","Type":"ContainerDied","Data":"388cf131bfb45e1e4ee31be5febbdf028f2bc915c425b77d1de9956b6f9594ff"}
Sep 09 12:21:54.422332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:54.422004    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d4699c9-4bfpg" event={"ID":"65f6bfd9-bb3c-47ed-8e53-55fa5884e064","Type":"ContainerDied","Data":"980e2c6361fac6d8c4b52300c2bbf6b231e90a3aaaf51591bfe689319ff9e77b"}
Sep 09 12:21:54.422332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:54.422031    2582 scope.go:117] "RemoveContainer" containerID="388cf131bfb45e1e4ee31be5febbdf028f2bc915c425b77d1de9956b6f9594ff"
Sep 09 12:21:54.430662 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:54.430648    2582 scope.go:117] "RemoveContainer" containerID="388cf131bfb45e1e4ee31be5febbdf028f2bc915c425b77d1de9956b6f9594ff"
Sep 09 12:21:54.431043 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:21:54.431018    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"388cf131bfb45e1e4ee31be5febbdf028f2bc915c425b77d1de9956b6f9594ff\": container with ID starting with 388cf131bfb45e1e4ee31be5febbdf028f2bc915c425b77d1de9956b6f9594ff not found: ID does not exist" containerID="388cf131bfb45e1e4ee31be5febbdf028f2bc915c425b77d1de9956b6f9594ff"
Sep 09 12:21:54.431131 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:54.431052    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"388cf131bfb45e1e4ee31be5febbdf028f2bc915c425b77d1de9956b6f9594ff"} err="failed to get container status \"388cf131bfb45e1e4ee31be5febbdf028f2bc915c425b77d1de9956b6f9594ff\": rpc error: code = NotFound desc = could not find container \"388cf131bfb45e1e4ee31be5febbdf028f2bc915c425b77d1de9956b6f9594ff\": container with ID starting with 388cf131bfb45e1e4ee31be5febbdf028f2bc915c425b77d1de9956b6f9594ff not found: ID does not exist"
Sep 09 12:21:54.443914 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:54.443885    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67d4699c9-4bfpg"]
Sep 09 12:21:54.445677 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:54.445653    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67d4699c9-4bfpg"]
Sep 09 12:21:54.823700 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:21:54.823669    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65f6bfd9-bb3c-47ed-8e53-55fa5884e064" path="/var/lib/kubelet/pods/65f6bfd9-bb3c-47ed-8e53-55fa5884e064/volumes"
Sep 09 12:22:25.614400 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.614364    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"]
Sep 09 12:22:25.615025 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.614754    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65f6bfd9-bb3c-47ed-8e53-55fa5884e064" containerName="console"
Sep 09 12:22:25.615025 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.614772    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="65f6bfd9-bb3c-47ed-8e53-55fa5884e064" containerName="console"
Sep 09 12:22:25.615025 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.614866    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="65f6bfd9-bb3c-47ed-8e53-55fa5884e064" containerName="console"
Sep 09 12:22:25.617222 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.617201    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.620184 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.620159    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 09 12:22:25.620308 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.620252    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 09 12:22:25.620308 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.620280    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 09 12:22:25.620308 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.620292    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 09 12:22:25.620468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.620233    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 09 12:22:25.620468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.620224    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-45ncz\""
Sep 09 12:22:25.630039 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.626772    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 09 12:22:25.630039 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.629962    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"]
Sep 09 12:22:25.687543 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.687509    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45460890-41ca-41f5-b1a8-4aa4207d23d9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.687695 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.687550    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45460890-41ca-41f5-b1a8-4aa4207d23d9-serving-certs-ca-bundle\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.687695 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.687579    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45460890-41ca-41f5-b1a8-4aa4207d23d9-metrics-client-ca\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.687695 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.687608    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/45460890-41ca-41f5-b1a8-4aa4207d23d9-federate-client-tls\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.687695 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.687636    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph8x5\" (UniqueName: \"kubernetes.io/projected/45460890-41ca-41f5-b1a8-4aa4207d23d9-kube-api-access-ph8x5\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.687695 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.687667    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/45460890-41ca-41f5-b1a8-4aa4207d23d9-telemeter-client-tls\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.688068 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.687711    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/45460890-41ca-41f5-b1a8-4aa4207d23d9-secret-telemeter-client\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.688068 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.687748    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45460890-41ca-41f5-b1a8-4aa4207d23d9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.789107 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.789057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45460890-41ca-41f5-b1a8-4aa4207d23d9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.789274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.789123    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45460890-41ca-41f5-b1a8-4aa4207d23d9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.789274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.789167    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45460890-41ca-41f5-b1a8-4aa4207d23d9-serving-certs-ca-bundle\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.789274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.789201    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45460890-41ca-41f5-b1a8-4aa4207d23d9-metrics-client-ca\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.789274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.789237    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/45460890-41ca-41f5-b1a8-4aa4207d23d9-federate-client-tls\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.789465 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.789278    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph8x5\" (UniqueName: \"kubernetes.io/projected/45460890-41ca-41f5-b1a8-4aa4207d23d9-kube-api-access-ph8x5\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.789702 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.789658    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/45460890-41ca-41f5-b1a8-4aa4207d23d9-telemeter-client-tls\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.789838 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.789717    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/45460890-41ca-41f5-b1a8-4aa4207d23d9-secret-telemeter-client\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.790061 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.790032    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45460890-41ca-41f5-b1a8-4aa4207d23d9-serving-certs-ca-bundle\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.790228 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.790194    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45460890-41ca-41f5-b1a8-4aa4207d23d9-metrics-client-ca\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.790567 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.790538    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45460890-41ca-41f5-b1a8-4aa4207d23d9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.792109 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.792083    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45460890-41ca-41f5-b1a8-4aa4207d23d9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.792221 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.792093    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/45460890-41ca-41f5-b1a8-4aa4207d23d9-secret-telemeter-client\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.792436 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.792415    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/45460890-41ca-41f5-b1a8-4aa4207d23d9-federate-client-tls\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.792734 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.792711    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/45460890-41ca-41f5-b1a8-4aa4207d23d9-telemeter-client-tls\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.797931 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.797905    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph8x5\" (UniqueName: \"kubernetes.io/projected/45460890-41ca-41f5-b1a8-4aa4207d23d9-kube-api-access-ph8x5\") pod \"telemeter-client-7799b8d7d9-kzb2w\" (UID: \"45460890-41ca-41f5-b1a8-4aa4207d23d9\") " pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:25.934374 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:25.934022    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"
Sep 09 12:22:26.222903 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:26.222880    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w"]
Sep 09 12:22:26.225402 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:22:26.225348    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45460890_41ca_41f5_b1a8_4aa4207d23d9.slice/crio-6e702fba72f6453217cce25df853f61283af1a81564fc5c13b7c5a48c5275421 WatchSource:0}: Error finding container 6e702fba72f6453217cce25df853f61283af1a81564fc5c13b7c5a48c5275421: Status 404 returned error can't find the container with id 6e702fba72f6453217cce25df853f61283af1a81564fc5c13b7c5a48c5275421
Sep 09 12:22:26.544051 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:26.544017    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w" event={"ID":"45460890-41ca-41f5-b1a8-4aa4207d23d9","Type":"ContainerStarted","Data":"6e702fba72f6453217cce25df853f61283af1a81564fc5c13b7c5a48c5275421"}
Sep 09 12:22:28.553861 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:28.553819    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w" event={"ID":"45460890-41ca-41f5-b1a8-4aa4207d23d9","Type":"ContainerStarted","Data":"adf3df0fd112bf2a0c2d8aecc343d46cab2714d711572ab4005184b56b91160c"}
Sep 09 12:22:29.560072 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.559993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w" event={"ID":"45460890-41ca-41f5-b1a8-4aa4207d23d9","Type":"ContainerStarted","Data":"908ad339fabe231d69dc3467315b914068522220ac73708a3b4654adb9514cbc"}
Sep 09 12:22:29.560072 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.560048    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w" event={"ID":"45460890-41ca-41f5-b1a8-4aa4207d23d9","Type":"ContainerStarted","Data":"4deeb22656623140e591dad60f923b3130d644c1a4f9df761d07b133ae3084fa"}
Sep 09 12:22:29.582954 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.582897    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7799b8d7d9-kzb2w" podStartSLOduration=1.576429157 podStartE2EDuration="4.582828953s" totalImagesPullingTime="3.006399796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:22:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:22:29.581930695 +0000 UTC m=+169.278495858" watchObservedRunningTime="2026-09-09 12:22:29.582828953 +0000 UTC m=+169.279394122"
Sep 09 12:22:29.674438 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.674401    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bffbd7f5-smkn9"]
Sep 09 12:22:29.676750 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.676721    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.687364 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.687340    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bffbd7f5-smkn9"]
Sep 09 12:22:29.827620 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.827533    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-oauth-serving-cert\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.827620 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.827576    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-config\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.827620 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.827593    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-serving-cert\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.827914 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.827653    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-trusted-ca-bundle\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.827914 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.827695    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-oauth-config\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.827914 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.827751    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-service-ca\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.827914 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.827795    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-262kq\" (UniqueName: \"kubernetes.io/projected/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-kube-api-access-262kq\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.928069 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.928044    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-oauth-serving-cert\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.928166 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.928089    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-config\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.928166 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.928108    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-serving-cert\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.928278 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.928243    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-trusted-ca-bundle\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.928334 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.928295    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-oauth-config\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.928755 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.928722    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-service-ca\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.929048 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.928991    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-262kq\" (UniqueName: \"kubernetes.io/projected/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-kube-api-access-262kq\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.929048 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.929055    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-config\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.929048 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.929055    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-oauth-serving-cert\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.929297 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.929218    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-trusted-ca-bundle\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.929458 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.929434    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-service-ca\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.930705 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.930688    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-serving-cert\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.930808 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.930790    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-oauth-config\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.937131 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.937113    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-262kq\" (UniqueName: \"kubernetes.io/projected/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-kube-api-access-262kq\") pod \"console-6bffbd7f5-smkn9\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") " pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:29.986010 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:29.985983    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:30.253973 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:30.253952    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bffbd7f5-smkn9"]
Sep 09 12:22:30.256394 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:22:30.256373    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbd75dc7_8421_4bc1_8b74_2268d129fbeb.slice/crio-0c48276b9966a5539dc4dc503d28e1eb971ec5ad518c2c7eb7d98bd1451beb18 WatchSource:0}: Error finding container 0c48276b9966a5539dc4dc503d28e1eb971ec5ad518c2c7eb7d98bd1451beb18: Status 404 returned error can't find the container with id 0c48276b9966a5539dc4dc503d28e1eb971ec5ad518c2c7eb7d98bd1451beb18
Sep 09 12:22:30.565306 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:30.565221    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bffbd7f5-smkn9" event={"ID":"cbd75dc7-8421-4bc1-8b74-2268d129fbeb","Type":"ContainerStarted","Data":"ed4e06f06c908342c08fffc6e9a3de516e324ffa92310eff801f3f3ee4ff9209"}
Sep 09 12:22:30.565306 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:30.565282    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bffbd7f5-smkn9" event={"ID":"cbd75dc7-8421-4bc1-8b74-2268d129fbeb","Type":"ContainerStarted","Data":"0c48276b9966a5539dc4dc503d28e1eb971ec5ad518c2c7eb7d98bd1451beb18"}
Sep 09 12:22:30.583609 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:30.583564    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bffbd7f5-smkn9" podStartSLOduration=1.58354756 podStartE2EDuration="1.58354756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:22:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:22:30.58232266 +0000 UTC m=+170.278887813" watchObservedRunningTime="2026-09-09 12:22:30.58354756 +0000 UTC m=+170.280112712"
Sep 09 12:22:39.986631 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:39.986546    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:39.987004 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:39.986651    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:39.991349 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:39.991325    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:40.609325 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:40.609296    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:22:40.655121 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:22:40.655085    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59c7f676cf-zq5pq"]
Sep 09 12:23:05.680588 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.680547    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59c7f676cf-zq5pq" podUID="4d511574-c055-44a4-bf8a-260265e399b8" containerName="console" containerID="cri-o://66bacdcdd44133be767082926cfd7aca7adc8b0e5a1d3c9b006fd2de503b7f49" gracePeriod=15
Sep 09 12:23:05.915648 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.915619    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:23:05.994705 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.994682    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d511574-c055-44a4-bf8a-260265e399b8-console-serving-cert\") pod \"4d511574-c055-44a4-bf8a-260265e399b8\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") "
Sep 09 12:23:05.994833 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.994716    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-trusted-ca-bundle\") pod \"4d511574-c055-44a4-bf8a-260265e399b8\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") "
Sep 09 12:23:05.994833 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.994735    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-console-config\") pod \"4d511574-c055-44a4-bf8a-260265e399b8\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") "
Sep 09 12:23:05.994833 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.994755    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6lsr\" (UniqueName: \"kubernetes.io/projected/4d511574-c055-44a4-bf8a-260265e399b8-kube-api-access-b6lsr\") pod \"4d511574-c055-44a4-bf8a-260265e399b8\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") "
Sep 09 12:23:05.994833 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.994808    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-service-ca\") pod \"4d511574-c055-44a4-bf8a-260265e399b8\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") "
Sep 09 12:23:05.995061 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.994835    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d511574-c055-44a4-bf8a-260265e399b8-console-oauth-config\") pod \"4d511574-c055-44a4-bf8a-260265e399b8\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") "
Sep 09 12:23:05.995061 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.994893    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-oauth-serving-cert\") pod \"4d511574-c055-44a4-bf8a-260265e399b8\" (UID: \"4d511574-c055-44a4-bf8a-260265e399b8\") "
Sep 09 12:23:05.995262 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.995234    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-console-config" (OuterVolumeSpecName: "console-config") pod "4d511574-c055-44a4-bf8a-260265e399b8" (UID: "4d511574-c055-44a4-bf8a-260265e399b8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:23:05.995337 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.995232    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4d511574-c055-44a4-bf8a-260265e399b8" (UID: "4d511574-c055-44a4-bf8a-260265e399b8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:23:05.995337 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.995291    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-service-ca" (OuterVolumeSpecName: "service-ca") pod "4d511574-c055-44a4-bf8a-260265e399b8" (UID: "4d511574-c055-44a4-bf8a-260265e399b8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:23:05.995446 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.995322    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4d511574-c055-44a4-bf8a-260265e399b8" (UID: "4d511574-c055-44a4-bf8a-260265e399b8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:23:05.997103 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.997081    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d511574-c055-44a4-bf8a-260265e399b8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4d511574-c055-44a4-bf8a-260265e399b8" (UID: "4d511574-c055-44a4-bf8a-260265e399b8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:23:05.997232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.997215    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d511574-c055-44a4-bf8a-260265e399b8-kube-api-access-b6lsr" (OuterVolumeSpecName: "kube-api-access-b6lsr") pod "4d511574-c055-44a4-bf8a-260265e399b8" (UID: "4d511574-c055-44a4-bf8a-260265e399b8"). InnerVolumeSpecName "kube-api-access-b6lsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:23:05.997280 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:05.997221    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d511574-c055-44a4-bf8a-260265e399b8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4d511574-c055-44a4-bf8a-260265e399b8" (UID: "4d511574-c055-44a4-bf8a-260265e399b8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:23:06.096119 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.096096    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:23:06.096119 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.096115    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d511574-c055-44a4-bf8a-260265e399b8-console-oauth-config\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:23:06.096239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.096124    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-oauth-serving-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:23:06.096239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.096132    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d511574-c055-44a4-bf8a-260265e399b8-console-serving-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:23:06.096239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.096140    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-trusted-ca-bundle\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:23:06.096239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.096149    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d511574-c055-44a4-bf8a-260265e399b8-console-config\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:23:06.096239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.096157    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6lsr\" (UniqueName: \"kubernetes.io/projected/4d511574-c055-44a4-bf8a-260265e399b8-kube-api-access-b6lsr\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:23:06.701291 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.701259    2582 generic.go:356] "Generic (PLEG): container finished" podID="4d511574-c055-44a4-bf8a-260265e399b8" containerID="66bacdcdd44133be767082926cfd7aca7adc8b0e5a1d3c9b006fd2de503b7f49" exitCode=0
Sep 09 12:23:06.701670 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.701304    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c7f676cf-zq5pq"
Sep 09 12:23:06.701670 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.701342    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c7f676cf-zq5pq" event={"ID":"4d511574-c055-44a4-bf8a-260265e399b8","Type":"ContainerDied","Data":"66bacdcdd44133be767082926cfd7aca7adc8b0e5a1d3c9b006fd2de503b7f49"}
Sep 09 12:23:06.701670 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.701395    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c7f676cf-zq5pq" event={"ID":"4d511574-c055-44a4-bf8a-260265e399b8","Type":"ContainerDied","Data":"25b719aa6d2370195e694a52920a863285a88db91fa56db5b7c41bc2161ac67a"}
Sep 09 12:23:06.701670 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.701415    2582 scope.go:117] "RemoveContainer" containerID="66bacdcdd44133be767082926cfd7aca7adc8b0e5a1d3c9b006fd2de503b7f49"
Sep 09 12:23:06.709905 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.709890    2582 scope.go:117] "RemoveContainer" containerID="66bacdcdd44133be767082926cfd7aca7adc8b0e5a1d3c9b006fd2de503b7f49"
Sep 09 12:23:06.710171 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:23:06.710155    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66bacdcdd44133be767082926cfd7aca7adc8b0e5a1d3c9b006fd2de503b7f49\": container with ID starting with 66bacdcdd44133be767082926cfd7aca7adc8b0e5a1d3c9b006fd2de503b7f49 not found: ID does not exist" containerID="66bacdcdd44133be767082926cfd7aca7adc8b0e5a1d3c9b006fd2de503b7f49"
Sep 09 12:23:06.710223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.710177    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66bacdcdd44133be767082926cfd7aca7adc8b0e5a1d3c9b006fd2de503b7f49"} err="failed to get container status \"66bacdcdd44133be767082926cfd7aca7adc8b0e5a1d3c9b006fd2de503b7f49\": rpc error: code = NotFound desc = could not find container \"66bacdcdd44133be767082926cfd7aca7adc8b0e5a1d3c9b006fd2de503b7f49\": container with ID starting with 66bacdcdd44133be767082926cfd7aca7adc8b0e5a1d3c9b006fd2de503b7f49 not found: ID does not exist"
Sep 09 12:23:06.721911 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.721883    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59c7f676cf-zq5pq"]
Sep 09 12:23:06.723595 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.723576    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59c7f676cf-zq5pq"]
Sep 09 12:23:06.823413 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:06.823380    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d511574-c055-44a4-bf8a-260265e399b8" path="/var/lib/kubelet/pods/4d511574-c055-44a4-bf8a-260265e399b8/volumes"
Sep 09 12:23:58.981767 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:58.981738    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-556fd77f69-v5z8d"]
Sep 09 12:23:58.982165 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:58.982021    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d511574-c055-44a4-bf8a-260265e399b8" containerName="console"
Sep 09 12:23:58.982165 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:58.982033    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d511574-c055-44a4-bf8a-260265e399b8" containerName="console"
Sep 09 12:23:58.982165 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:58.982083    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d511574-c055-44a4-bf8a-260265e399b8" containerName="console"
Sep 09 12:23:58.986062 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:58.986045    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.002203 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.002174    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-556fd77f69-v5z8d"]
Sep 09 12:23:59.064069 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.064044    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-oauth-serving-cert\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.064183 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.064080    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59f89\" (UniqueName: \"kubernetes.io/projected/6e744d90-e9d2-4ac9-b9dd-936a162ac888-kube-api-access-59f89\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.064183 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.064118    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-oauth-config\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.064183 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.064151    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-serving-cert\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.064183 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.064169    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-service-ca\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.064316 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.064195    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-config\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.064316 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.064226    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-trusted-ca-bundle\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.164750 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.164725    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-oauth-config\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.164893 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.164763    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-serving-cert\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.164893 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.164784    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-service-ca\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.164893 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.164803    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-config\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.164893 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.164829    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-trusted-ca-bundle\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.165085 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.164897    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-oauth-serving-cert\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.165085 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.164917    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59f89\" (UniqueName: \"kubernetes.io/projected/6e744d90-e9d2-4ac9-b9dd-936a162ac888-kube-api-access-59f89\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.165652 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.165566    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-service-ca\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.165652 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.165625    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-config\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.165652 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.165625    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-oauth-serving-cert\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.165912 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.165829    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-trusted-ca-bundle\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.167417 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.167393    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-oauth-config\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.167500 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.167476    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-serving-cert\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.172128 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.172108    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59f89\" (UniqueName: \"kubernetes.io/projected/6e744d90-e9d2-4ac9-b9dd-936a162ac888-kube-api-access-59f89\") pod \"console-556fd77f69-v5z8d\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") " pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.294352 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.294320    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:23:59.569015 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.568945    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-556fd77f69-v5z8d"]
Sep 09 12:23:59.572182 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:23:59.572151    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e744d90_e9d2_4ac9_b9dd_936a162ac888.slice/crio-7ce0ed7eac52c9d88371825e236b55c2d88db38bef1a2662b516a02b70c5ba25 WatchSource:0}: Error finding container 7ce0ed7eac52c9d88371825e236b55c2d88db38bef1a2662b516a02b70c5ba25: Status 404 returned error can't find the container with id 7ce0ed7eac52c9d88371825e236b55c2d88db38bef1a2662b516a02b70c5ba25
Sep 09 12:23:59.898174 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.898089    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556fd77f69-v5z8d" event={"ID":"6e744d90-e9d2-4ac9-b9dd-936a162ac888","Type":"ContainerStarted","Data":"2b9371b1d76f680584e3c4071042c4c4f32ffea5cf1d540fc8dd6b104a45e047"}
Sep 09 12:23:59.898174 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.898142    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556fd77f69-v5z8d" event={"ID":"6e744d90-e9d2-4ac9-b9dd-936a162ac888","Type":"ContainerStarted","Data":"7ce0ed7eac52c9d88371825e236b55c2d88db38bef1a2662b516a02b70c5ba25"}
Sep 09 12:23:59.917563 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:23:59.917515    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-556fd77f69-v5z8d" podStartSLOduration=1.917498985 podStartE2EDuration="1.917498985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:23:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:23:59.916143592 +0000 UTC m=+259.612708744" watchObservedRunningTime="2026-09-09 12:23:59.917498985 +0000 UTC m=+259.614064173"
Sep 09 12:24:09.294723 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:09.294650    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:24:09.294723 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:09.294716    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:24:09.299289 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:09.299270    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:24:09.934571 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:09.934541    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:24:09.982591 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:09.982563    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bffbd7f5-smkn9"]
Sep 09 12:24:35.004693 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.004650    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6bffbd7f5-smkn9" podUID="cbd75dc7-8421-4bc1-8b74-2268d129fbeb" containerName="console" containerID="cri-o://ed4e06f06c908342c08fffc6e9a3de516e324ffa92310eff801f3f3ee4ff9209" gracePeriod=15
Sep 09 12:24:35.238487 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.238466    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:24:35.417876 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.417799    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-service-ca\") pod \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") "
Sep 09 12:24:35.417876 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.417828    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-serving-cert\") pod \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") "
Sep 09 12:24:35.417876 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.417843    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-trusted-ca-bundle\") pod \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") "
Sep 09 12:24:35.418080 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.417997    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-oauth-config\") pod \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") "
Sep 09 12:24:35.418128 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.418076    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-262kq\" (UniqueName: \"kubernetes.io/projected/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-kube-api-access-262kq\") pod \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") "
Sep 09 12:24:35.418128 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.418122    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-oauth-serving-cert\") pod \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") "
Sep 09 12:24:35.418217 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.418160    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-config\") pod \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\" (UID: \"cbd75dc7-8421-4bc1-8b74-2268d129fbeb\") "
Sep 09 12:24:35.418217 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.418190    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-service-ca" (OuterVolumeSpecName: "service-ca") pod "cbd75dc7-8421-4bc1-8b74-2268d129fbeb" (UID: "cbd75dc7-8421-4bc1-8b74-2268d129fbeb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:24:35.418317 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.418235    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cbd75dc7-8421-4bc1-8b74-2268d129fbeb" (UID: "cbd75dc7-8421-4bc1-8b74-2268d129fbeb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:24:35.418375 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.418350    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:24:35.418375 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.418364    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-trusted-ca-bundle\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:24:35.418522 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.418498    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cbd75dc7-8421-4bc1-8b74-2268d129fbeb" (UID: "cbd75dc7-8421-4bc1-8b74-2268d129fbeb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:24:35.418591 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.418524    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-config" (OuterVolumeSpecName: "console-config") pod "cbd75dc7-8421-4bc1-8b74-2268d129fbeb" (UID: "cbd75dc7-8421-4bc1-8b74-2268d129fbeb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:24:35.420096 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.420072    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cbd75dc7-8421-4bc1-8b74-2268d129fbeb" (UID: "cbd75dc7-8421-4bc1-8b74-2268d129fbeb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:24:35.420206 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.420111    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cbd75dc7-8421-4bc1-8b74-2268d129fbeb" (UID: "cbd75dc7-8421-4bc1-8b74-2268d129fbeb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:24:35.420260 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.420199    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-kube-api-access-262kq" (OuterVolumeSpecName: "kube-api-access-262kq") pod "cbd75dc7-8421-4bc1-8b74-2268d129fbeb" (UID: "cbd75dc7-8421-4bc1-8b74-2268d129fbeb"). InnerVolumeSpecName "kube-api-access-262kq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:24:35.519168 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.519143    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-262kq\" (UniqueName: \"kubernetes.io/projected/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-kube-api-access-262kq\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:24:35.519168 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.519163    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-oauth-serving-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:24:35.519168 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.519171    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-config\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:24:35.519317 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.519179    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-serving-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:24:35.519317 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:35.519187    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cbd75dc7-8421-4bc1-8b74-2268d129fbeb-console-oauth-config\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:24:36.022693 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:36.022660    2582 generic.go:356] "Generic (PLEG): container finished" podID="cbd75dc7-8421-4bc1-8b74-2268d129fbeb" containerID="ed4e06f06c908342c08fffc6e9a3de516e324ffa92310eff801f3f3ee4ff9209" exitCode=0
Sep 09 12:24:36.023124 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:36.022702    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bffbd7f5-smkn9"
Sep 09 12:24:36.023124 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:36.022730    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bffbd7f5-smkn9" event={"ID":"cbd75dc7-8421-4bc1-8b74-2268d129fbeb","Type":"ContainerDied","Data":"ed4e06f06c908342c08fffc6e9a3de516e324ffa92310eff801f3f3ee4ff9209"}
Sep 09 12:24:36.023124 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:36.022773    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bffbd7f5-smkn9" event={"ID":"cbd75dc7-8421-4bc1-8b74-2268d129fbeb","Type":"ContainerDied","Data":"0c48276b9966a5539dc4dc503d28e1eb971ec5ad518c2c7eb7d98bd1451beb18"}
Sep 09 12:24:36.023124 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:36.022797    2582 scope.go:117] "RemoveContainer" containerID="ed4e06f06c908342c08fffc6e9a3de516e324ffa92310eff801f3f3ee4ff9209"
Sep 09 12:24:36.032506 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:36.032485    2582 scope.go:117] "RemoveContainer" containerID="ed4e06f06c908342c08fffc6e9a3de516e324ffa92310eff801f3f3ee4ff9209"
Sep 09 12:24:36.032925 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:24:36.032797    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed4e06f06c908342c08fffc6e9a3de516e324ffa92310eff801f3f3ee4ff9209\": container with ID starting with ed4e06f06c908342c08fffc6e9a3de516e324ffa92310eff801f3f3ee4ff9209 not found: ID does not exist" containerID="ed4e06f06c908342c08fffc6e9a3de516e324ffa92310eff801f3f3ee4ff9209"
Sep 09 12:24:36.032925 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:36.032831    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed4e06f06c908342c08fffc6e9a3de516e324ffa92310eff801f3f3ee4ff9209"} err="failed to get container status \"ed4e06f06c908342c08fffc6e9a3de516e324ffa92310eff801f3f3ee4ff9209\": rpc error: code = NotFound desc = could not find container \"ed4e06f06c908342c08fffc6e9a3de516e324ffa92310eff801f3f3ee4ff9209\": container with ID starting with ed4e06f06c908342c08fffc6e9a3de516e324ffa92310eff801f3f3ee4ff9209 not found: ID does not exist"
Sep 09 12:24:36.046044 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:36.046023    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bffbd7f5-smkn9"]
Sep 09 12:24:36.050130 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:36.050112    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bffbd7f5-smkn9"]
Sep 09 12:24:36.823883 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:36.823833    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbd75dc7-8421-4bc1-8b74-2268d129fbeb" path="/var/lib/kubelet/pods/cbd75dc7-8421-4bc1-8b74-2268d129fbeb/volumes"
Sep 09 12:24:40.739002 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:40.738978    2582 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 12:24:45.695866 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.695819    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"]
Sep 09 12:24:45.698230 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.696109    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbd75dc7-8421-4bc1-8b74-2268d129fbeb" containerName="console"
Sep 09 12:24:45.698230 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.696121    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbd75dc7-8421-4bc1-8b74-2268d129fbeb" containerName="console"
Sep 09 12:24:45.698230 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.696163    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbd75dc7-8421-4bc1-8b74-2268d129fbeb" containerName="console"
Sep 09 12:24:45.699066 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.699052    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"
Sep 09 12:24:45.701273 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.701253    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 12:24:45.701401 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.701285    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sghgc\""
Sep 09 12:24:45.701401 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.701253    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 12:24:45.708639 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.708619    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"]
Sep 09 12:24:45.786895 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.786869    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dxf4\" (UniqueName: \"kubernetes.io/projected/67e75b64-ddcf-4740-938e-5d348b2f37ee-kube-api-access-9dxf4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87\" (UID: \"67e75b64-ddcf-4740-938e-5d348b2f37ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"
Sep 09 12:24:45.787043 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.786910    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67e75b64-ddcf-4740-938e-5d348b2f37ee-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87\" (UID: \"67e75b64-ddcf-4740-938e-5d348b2f37ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"
Sep 09 12:24:45.787043 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.786980    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67e75b64-ddcf-4740-938e-5d348b2f37ee-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87\" (UID: \"67e75b64-ddcf-4740-938e-5d348b2f37ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"
Sep 09 12:24:45.887640 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.887604    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dxf4\" (UniqueName: \"kubernetes.io/projected/67e75b64-ddcf-4740-938e-5d348b2f37ee-kube-api-access-9dxf4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87\" (UID: \"67e75b64-ddcf-4740-938e-5d348b2f37ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"
Sep 09 12:24:45.887752 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.887654    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67e75b64-ddcf-4740-938e-5d348b2f37ee-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87\" (UID: \"67e75b64-ddcf-4740-938e-5d348b2f37ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"
Sep 09 12:24:45.887752 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.887695    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67e75b64-ddcf-4740-938e-5d348b2f37ee-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87\" (UID: \"67e75b64-ddcf-4740-938e-5d348b2f37ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"
Sep 09 12:24:45.888055 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.888038    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67e75b64-ddcf-4740-938e-5d348b2f37ee-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87\" (UID: \"67e75b64-ddcf-4740-938e-5d348b2f37ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"
Sep 09 12:24:45.888105 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.888069    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67e75b64-ddcf-4740-938e-5d348b2f37ee-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87\" (UID: \"67e75b64-ddcf-4740-938e-5d348b2f37ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"
Sep 09 12:24:45.896982 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:45.896961    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dxf4\" (UniqueName: \"kubernetes.io/projected/67e75b64-ddcf-4740-938e-5d348b2f37ee-kube-api-access-9dxf4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87\" (UID: \"67e75b64-ddcf-4740-938e-5d348b2f37ee\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"
Sep 09 12:24:46.007777 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:46.007758    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"
Sep 09 12:24:46.278095 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:46.278073    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"]
Sep 09 12:24:46.280343 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:24:46.280308    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e75b64_ddcf_4740_938e_5d348b2f37ee.slice/crio-74deb7895fc5d6eee506d9e5047fd1e596f164665952977249c7a5f560623d6f WatchSource:0}: Error finding container 74deb7895fc5d6eee506d9e5047fd1e596f164665952977249c7a5f560623d6f: Status 404 returned error can't find the container with id 74deb7895fc5d6eee506d9e5047fd1e596f164665952977249c7a5f560623d6f
Sep 09 12:24:46.282232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:46.282216    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:24:47.065758 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:47.065723    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87" event={"ID":"67e75b64-ddcf-4740-938e-5d348b2f37ee","Type":"ContainerStarted","Data":"74deb7895fc5d6eee506d9e5047fd1e596f164665952977249c7a5f560623d6f"}
Sep 09 12:24:52.087993 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:52.087953    2582 generic.go:356] "Generic (PLEG): container finished" podID="67e75b64-ddcf-4740-938e-5d348b2f37ee" containerID="02e816c85b2c1be605d922e8902ea61cadedd38d7d5b1a20a73f2d6291e0b32e" exitCode=0
Sep 09 12:24:52.088367 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:52.088000    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87" event={"ID":"67e75b64-ddcf-4740-938e-5d348b2f37ee","Type":"ContainerDied","Data":"02e816c85b2c1be605d922e8902ea61cadedd38d7d5b1a20a73f2d6291e0b32e"}
Sep 09 12:24:54.097916 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:54.097887    2582 generic.go:356] "Generic (PLEG): container finished" podID="67e75b64-ddcf-4740-938e-5d348b2f37ee" containerID="a57fa09eeecc0e4a43998c1924eb05e0f4e48e87280a879eb673af6df27b2d13" exitCode=0
Sep 09 12:24:54.098274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:24:54.097920    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87" event={"ID":"67e75b64-ddcf-4740-938e-5d348b2f37ee","Type":"ContainerDied","Data":"a57fa09eeecc0e4a43998c1924eb05e0f4e48e87280a879eb673af6df27b2d13"}
Sep 09 12:25:01.124803 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:01.124772    2582 generic.go:356] "Generic (PLEG): container finished" podID="67e75b64-ddcf-4740-938e-5d348b2f37ee" containerID="dbe52a37eff31100a32e1b3b69284a131e4ec088aab5f74f732350b4f679625e" exitCode=0
Sep 09 12:25:01.125152 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:01.124873    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87" event={"ID":"67e75b64-ddcf-4740-938e-5d348b2f37ee","Type":"ContainerDied","Data":"dbe52a37eff31100a32e1b3b69284a131e4ec088aab5f74f732350b4f679625e"}
Sep 09 12:25:02.249744 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:02.249722    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"
Sep 09 12:25:02.317221 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:02.317195    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67e75b64-ddcf-4740-938e-5d348b2f37ee-bundle\") pod \"67e75b64-ddcf-4740-938e-5d348b2f37ee\" (UID: \"67e75b64-ddcf-4740-938e-5d348b2f37ee\") "
Sep 09 12:25:02.317351 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:02.317260    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67e75b64-ddcf-4740-938e-5d348b2f37ee-util\") pod \"67e75b64-ddcf-4740-938e-5d348b2f37ee\" (UID: \"67e75b64-ddcf-4740-938e-5d348b2f37ee\") "
Sep 09 12:25:02.317351 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:02.317312    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dxf4\" (UniqueName: \"kubernetes.io/projected/67e75b64-ddcf-4740-938e-5d348b2f37ee-kube-api-access-9dxf4\") pod \"67e75b64-ddcf-4740-938e-5d348b2f37ee\" (UID: \"67e75b64-ddcf-4740-938e-5d348b2f37ee\") "
Sep 09 12:25:02.317788 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:02.317763    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e75b64-ddcf-4740-938e-5d348b2f37ee-bundle" (OuterVolumeSpecName: "bundle") pod "67e75b64-ddcf-4740-938e-5d348b2f37ee" (UID: "67e75b64-ddcf-4740-938e-5d348b2f37ee"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:25:02.319535 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:02.319509    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67e75b64-ddcf-4740-938e-5d348b2f37ee-kube-api-access-9dxf4" (OuterVolumeSpecName: "kube-api-access-9dxf4") pod "67e75b64-ddcf-4740-938e-5d348b2f37ee" (UID: "67e75b64-ddcf-4740-938e-5d348b2f37ee"). InnerVolumeSpecName "kube-api-access-9dxf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:25:02.322037 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:02.322017    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e75b64-ddcf-4740-938e-5d348b2f37ee-util" (OuterVolumeSpecName: "util") pod "67e75b64-ddcf-4740-938e-5d348b2f37ee" (UID: "67e75b64-ddcf-4740-938e-5d348b2f37ee"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:25:02.417781 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:02.417735    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dxf4\" (UniqueName: \"kubernetes.io/projected/67e75b64-ddcf-4740-938e-5d348b2f37ee-kube-api-access-9dxf4\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:25:02.417781 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:02.417755    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67e75b64-ddcf-4740-938e-5d348b2f37ee-bundle\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:25:02.417781 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:02.417763    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67e75b64-ddcf-4740-938e-5d348b2f37ee-util\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:25:03.133282 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:03.133213    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87" event={"ID":"67e75b64-ddcf-4740-938e-5d348b2f37ee","Type":"ContainerDied","Data":"74deb7895fc5d6eee506d9e5047fd1e596f164665952977249c7a5f560623d6f"}
Sep 09 12:25:03.133282 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:03.133231    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dsk87"
Sep 09 12:25:03.133282 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:03.133263    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74deb7895fc5d6eee506d9e5047fd1e596f164665952977249c7a5f560623d6f"
Sep 09 12:25:08.374121 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.374090    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7clf6"]
Sep 09 12:25:08.374467 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.374342    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67e75b64-ddcf-4740-938e-5d348b2f37ee" containerName="extract"
Sep 09 12:25:08.374467 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.374353    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="67e75b64-ddcf-4740-938e-5d348b2f37ee" containerName="extract"
Sep 09 12:25:08.374467 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.374368    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67e75b64-ddcf-4740-938e-5d348b2f37ee" containerName="pull"
Sep 09 12:25:08.374467 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.374373    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="67e75b64-ddcf-4740-938e-5d348b2f37ee" containerName="pull"
Sep 09 12:25:08.374467 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.374383    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67e75b64-ddcf-4740-938e-5d348b2f37ee" containerName="util"
Sep 09 12:25:08.374467 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.374388    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="67e75b64-ddcf-4740-938e-5d348b2f37ee" containerName="util"
Sep 09 12:25:08.374467 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.374444    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="67e75b64-ddcf-4740-938e-5d348b2f37ee" containerName="extract"
Sep 09 12:25:08.426288 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.426262    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7clf6"]
Sep 09 12:25:08.426409 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.426353    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7clf6"
Sep 09 12:25:08.428874 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.428832    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:25:08.428999 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.428834    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-htdcz\""
Sep 09 12:25:08.428999 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.428902    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 09 12:25:08.460755 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.460718    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn4pm\" (UniqueName: \"kubernetes.io/projected/019d34a8-a7e4-496f-99d9-dad5b4f77d8d-kube-api-access-qn4pm\") pod \"cert-manager-operator-controller-manager-55c444bf86-7clf6\" (UID: \"019d34a8-a7e4-496f-99d9-dad5b4f77d8d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7clf6"
Sep 09 12:25:08.460755 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.460763    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/019d34a8-a7e4-496f-99d9-dad5b4f77d8d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7clf6\" (UID: \"019d34a8-a7e4-496f-99d9-dad5b4f77d8d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7clf6"
Sep 09 12:25:08.561715 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.561683    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn4pm\" (UniqueName: \"kubernetes.io/projected/019d34a8-a7e4-496f-99d9-dad5b4f77d8d-kube-api-access-qn4pm\") pod \"cert-manager-operator-controller-manager-55c444bf86-7clf6\" (UID: \"019d34a8-a7e4-496f-99d9-dad5b4f77d8d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7clf6"
Sep 09 12:25:08.561894 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.561729    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/019d34a8-a7e4-496f-99d9-dad5b4f77d8d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7clf6\" (UID: \"019d34a8-a7e4-496f-99d9-dad5b4f77d8d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7clf6"
Sep 09 12:25:08.562134 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.562108    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/019d34a8-a7e4-496f-99d9-dad5b4f77d8d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7clf6\" (UID: \"019d34a8-a7e4-496f-99d9-dad5b4f77d8d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7clf6"
Sep 09 12:25:08.570552 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.570531    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn4pm\" (UniqueName: \"kubernetes.io/projected/019d34a8-a7e4-496f-99d9-dad5b4f77d8d-kube-api-access-qn4pm\") pod \"cert-manager-operator-controller-manager-55c444bf86-7clf6\" (UID: \"019d34a8-a7e4-496f-99d9-dad5b4f77d8d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7clf6"
Sep 09 12:25:08.735197 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:08.735119    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7clf6"
Sep 09 12:25:09.028468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:09.028442    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7clf6"]
Sep 09 12:25:09.031122 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:25:09.031098    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod019d34a8_a7e4_496f_99d9_dad5b4f77d8d.slice/crio-609145f61d6f4897a9a55118191ba24247034dd859ef28c6d0c8cdb63ca9b057 WatchSource:0}: Error finding container 609145f61d6f4897a9a55118191ba24247034dd859ef28c6d0c8cdb63ca9b057: Status 404 returned error can't find the container with id 609145f61d6f4897a9a55118191ba24247034dd859ef28c6d0c8cdb63ca9b057
Sep 09 12:25:09.155407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:09.155368    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7clf6" event={"ID":"019d34a8-a7e4-496f-99d9-dad5b4f77d8d","Type":"ContainerStarted","Data":"609145f61d6f4897a9a55118191ba24247034dd859ef28c6d0c8cdb63ca9b057"}
Sep 09 12:25:11.165550 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:11.165519    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7clf6" event={"ID":"019d34a8-a7e4-496f-99d9-dad5b4f77d8d","Type":"ContainerStarted","Data":"3bcf538c2826eedfb5fdaeab99b0fb9f8c73096980fe36f24431d99621afaae2"}
Sep 09 12:25:11.185207 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:11.185159    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7clf6" podStartSLOduration=1.185236024 podStartE2EDuration="3.18513958s" totalImagesPullingTime="1.999903556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:25:11.183372827 +0000 UTC m=+330.879937979" watchObservedRunningTime="2026-09-09 12:25:11.18513958 +0000 UTC m=+330.881704732"
Sep 09 12:25:12.150845 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:12.150808    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bqmch"]
Sep 09 12:25:12.166177 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:12.166153    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bqmch"]
Sep 09 12:25:12.166564 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:12.166237    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bqmch"
Sep 09 12:25:12.168238 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:12.168220    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 09 12:25:12.169144 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:12.169125    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-gsxlx\""
Sep 09 12:25:12.169244 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:12.169164    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 09 12:25:12.292941 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:12.292920    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gwbh\" (UniqueName: \"kubernetes.io/projected/2b049bc2-6019-48c9-a867-34da93c0d861-kube-api-access-6gwbh\") pod \"cert-manager-webhook-6486ffd997-bqmch\" (UID: \"2b049bc2-6019-48c9-a867-34da93c0d861\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bqmch"
Sep 09 12:25:12.293047 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:12.292994    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b049bc2-6019-48c9-a867-34da93c0d861-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bqmch\" (UID: \"2b049bc2-6019-48c9-a867-34da93c0d861\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bqmch"
Sep 09 12:25:12.393693 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:12.393666    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gwbh\" (UniqueName: \"kubernetes.io/projected/2b049bc2-6019-48c9-a867-34da93c0d861-kube-api-access-6gwbh\") pod \"cert-manager-webhook-6486ffd997-bqmch\" (UID: \"2b049bc2-6019-48c9-a867-34da93c0d861\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bqmch"
Sep 09 12:25:12.393874 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:12.393711    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b049bc2-6019-48c9-a867-34da93c0d861-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bqmch\" (UID: \"2b049bc2-6019-48c9-a867-34da93c0d861\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bqmch"
Sep 09 12:25:12.401705 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:12.401646    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b049bc2-6019-48c9-a867-34da93c0d861-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bqmch\" (UID: \"2b049bc2-6019-48c9-a867-34da93c0d861\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bqmch"
Sep 09 12:25:12.401810 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:12.401792    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gwbh\" (UniqueName: \"kubernetes.io/projected/2b049bc2-6019-48c9-a867-34da93c0d861-kube-api-access-6gwbh\") pod \"cert-manager-webhook-6486ffd997-bqmch\" (UID: \"2b049bc2-6019-48c9-a867-34da93c0d861\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bqmch"
Sep 09 12:25:12.482837 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:12.482802    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bqmch"
Sep 09 12:25:12.772660 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:12.772486    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bqmch"]
Sep 09 12:25:12.775187 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:25:12.775151    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b049bc2_6019_48c9_a867_34da93c0d861.slice/crio-ebc8f2ce3a55f67bf5140baecb1a2bc698c51815d4f03e0120f97c964405655e WatchSource:0}: Error finding container ebc8f2ce3a55f67bf5140baecb1a2bc698c51815d4f03e0120f97c964405655e: Status 404 returned error can't find the container with id ebc8f2ce3a55f67bf5140baecb1a2bc698c51815d4f03e0120f97c964405655e
Sep 09 12:25:13.175461 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:13.175386    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bqmch" event={"ID":"2b049bc2-6019-48c9-a867-34da93c0d861","Type":"ContainerStarted","Data":"ebc8f2ce3a55f67bf5140baecb1a2bc698c51815d4f03e0120f97c964405655e"}
Sep 09 12:25:16.189756 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:16.189722    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bqmch" event={"ID":"2b049bc2-6019-48c9-a867-34da93c0d861","Type":"ContainerStarted","Data":"f8fd2c09467c9d1262842dc90d9fef799781686314667a31f217180526589b63"}
Sep 09 12:25:16.190171 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:16.189774    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bqmch"
Sep 09 12:25:16.206718 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:16.206669    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-bqmch" podStartSLOduration=1.536495445 podStartE2EDuration="4.206654055s" totalImagesPullingTime="2.67015861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:25:16.205255159 +0000 UTC m=+335.901820322" watchObservedRunningTime="2026-09-09 12:25:16.206654055 +0000 UTC m=+335.903219206"
Sep 09 12:25:18.659768 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:18.659736    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cfhsf"]
Sep 09 12:25:18.661734 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:18.661713    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cfhsf"
Sep 09 12:25:18.663560 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:18.663541    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-hmlsx\""
Sep 09 12:25:18.671289 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:18.671270    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cfhsf"]
Sep 09 12:25:18.739898 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:18.739871    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c2c694da-bd1b-427d-9b92-3fe9f5f14ed9-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cfhsf\" (UID: \"c2c694da-bd1b-427d-9b92-3fe9f5f14ed9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cfhsf"
Sep 09 12:25:18.739997 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:18.739907    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwt9q\" (UniqueName: \"kubernetes.io/projected/c2c694da-bd1b-427d-9b92-3fe9f5f14ed9-kube-api-access-lwt9q\") pod \"cert-manager-cainjector-559cdd8c87-cfhsf\" (UID: \"c2c694da-bd1b-427d-9b92-3fe9f5f14ed9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cfhsf"
Sep 09 12:25:18.841053 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:18.841009    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c2c694da-bd1b-427d-9b92-3fe9f5f14ed9-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cfhsf\" (UID: \"c2c694da-bd1b-427d-9b92-3fe9f5f14ed9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cfhsf"
Sep 09 12:25:18.841053 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:18.841063    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwt9q\" (UniqueName: \"kubernetes.io/projected/c2c694da-bd1b-427d-9b92-3fe9f5f14ed9-kube-api-access-lwt9q\") pod \"cert-manager-cainjector-559cdd8c87-cfhsf\" (UID: \"c2c694da-bd1b-427d-9b92-3fe9f5f14ed9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cfhsf"
Sep 09 12:25:18.848919 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:18.848890    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c2c694da-bd1b-427d-9b92-3fe9f5f14ed9-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cfhsf\" (UID: \"c2c694da-bd1b-427d-9b92-3fe9f5f14ed9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cfhsf"
Sep 09 12:25:18.849051 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:18.849034    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwt9q\" (UniqueName: \"kubernetes.io/projected/c2c694da-bd1b-427d-9b92-3fe9f5f14ed9-kube-api-access-lwt9q\") pod \"cert-manager-cainjector-559cdd8c87-cfhsf\" (UID: \"c2c694da-bd1b-427d-9b92-3fe9f5f14ed9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cfhsf"
Sep 09 12:25:18.971300 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:18.971237    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cfhsf"
Sep 09 12:25:19.238922 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:19.238897    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cfhsf"]
Sep 09 12:25:19.242248 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:25:19.242215    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2c694da_bd1b_427d_9b92_3fe9f5f14ed9.slice/crio-6bdfecbe34b4df9357a24fc73700ec4562115e1f0a3c2964b3d29fa8af1ca500 WatchSource:0}: Error finding container 6bdfecbe34b4df9357a24fc73700ec4562115e1f0a3c2964b3d29fa8af1ca500: Status 404 returned error can't find the container with id 6bdfecbe34b4df9357a24fc73700ec4562115e1f0a3c2964b3d29fa8af1ca500
Sep 09 12:25:20.207186 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:20.207150    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cfhsf" event={"ID":"c2c694da-bd1b-427d-9b92-3fe9f5f14ed9","Type":"ContainerStarted","Data":"fd600cd879fe63cd1bd64688a76f0cb232cdffc779b44060603e9f0258a2e334"}
Sep 09 12:25:20.207186 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:20.207192    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cfhsf" event={"ID":"c2c694da-bd1b-427d-9b92-3fe9f5f14ed9","Type":"ContainerStarted","Data":"6bdfecbe34b4df9357a24fc73700ec4562115e1f0a3c2964b3d29fa8af1ca500"}
Sep 09 12:25:20.224534 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:20.224486    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cfhsf" podStartSLOduration=2.224470039 podStartE2EDuration="2.224470039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:25:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:25:20.223055061 +0000 UTC m=+339.919620215" watchObservedRunningTime="2026-09-09 12:25:20.224470039 +0000 UTC m=+339.921035190"
Sep 09 12:25:22.195866 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:22.195820    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bqmch"
Sep 09 12:25:22.558086 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:22.558052    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fs9wk"]
Sep 09 12:25:22.560107 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:22.560092    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-fs9wk"
Sep 09 12:25:22.562425 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:22.562409    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-m6lzx\""
Sep 09 12:25:22.572200 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:22.572178    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fs9wk"]
Sep 09 12:25:22.675724 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:22.675683    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3fc51923-6217-44e7-a1a5-63e64a653d71-bound-sa-token\") pod \"cert-manager-55b4d954d5-fs9wk\" (UID: \"3fc51923-6217-44e7-a1a5-63e64a653d71\") " pod="cert-manager/cert-manager-55b4d954d5-fs9wk"
Sep 09 12:25:22.675923 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:22.675806    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsjnm\" (UniqueName: \"kubernetes.io/projected/3fc51923-6217-44e7-a1a5-63e64a653d71-kube-api-access-tsjnm\") pod \"cert-manager-55b4d954d5-fs9wk\" (UID: \"3fc51923-6217-44e7-a1a5-63e64a653d71\") " pod="cert-manager/cert-manager-55b4d954d5-fs9wk"
Sep 09 12:25:22.777185 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:22.777146    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsjnm\" (UniqueName: \"kubernetes.io/projected/3fc51923-6217-44e7-a1a5-63e64a653d71-kube-api-access-tsjnm\") pod \"cert-manager-55b4d954d5-fs9wk\" (UID: \"3fc51923-6217-44e7-a1a5-63e64a653d71\") " pod="cert-manager/cert-manager-55b4d954d5-fs9wk"
Sep 09 12:25:22.777185 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:22.777186    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3fc51923-6217-44e7-a1a5-63e64a653d71-bound-sa-token\") pod \"cert-manager-55b4d954d5-fs9wk\" (UID: \"3fc51923-6217-44e7-a1a5-63e64a653d71\") " pod="cert-manager/cert-manager-55b4d954d5-fs9wk"
Sep 09 12:25:22.786868 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:22.786818    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3fc51923-6217-44e7-a1a5-63e64a653d71-bound-sa-token\") pod \"cert-manager-55b4d954d5-fs9wk\" (UID: \"3fc51923-6217-44e7-a1a5-63e64a653d71\") " pod="cert-manager/cert-manager-55b4d954d5-fs9wk"
Sep 09 12:25:22.786981 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:22.786912    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsjnm\" (UniqueName: \"kubernetes.io/projected/3fc51923-6217-44e7-a1a5-63e64a653d71-kube-api-access-tsjnm\") pod \"cert-manager-55b4d954d5-fs9wk\" (UID: \"3fc51923-6217-44e7-a1a5-63e64a653d71\") " pod="cert-manager/cert-manager-55b4d954d5-fs9wk"
Sep 09 12:25:22.869714 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:22.869626    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-fs9wk"
Sep 09 12:25:23.142942 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:23.142828    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fs9wk"]
Sep 09 12:25:23.146079 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:25:23.146052    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fc51923_6217_44e7_a1a5_63e64a653d71.slice/crio-6ecf154a02f1280c8923f0cdad4737f6be2d691320fb57f3112a740dd294449f WatchSource:0}: Error finding container 6ecf154a02f1280c8923f0cdad4737f6be2d691320fb57f3112a740dd294449f: Status 404 returned error can't find the container with id 6ecf154a02f1280c8923f0cdad4737f6be2d691320fb57f3112a740dd294449f
Sep 09 12:25:23.223881 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:23.223827    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-fs9wk" event={"ID":"3fc51923-6217-44e7-a1a5-63e64a653d71","Type":"ContainerStarted","Data":"6e73d0b3d0fd16088d192b8a5e642795cfbb4f6397ac091729485fb6e9701b29"}
Sep 09 12:25:23.224374 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:23.223897    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-fs9wk" event={"ID":"3fc51923-6217-44e7-a1a5-63e64a653d71","Type":"ContainerStarted","Data":"6ecf154a02f1280c8923f0cdad4737f6be2d691320fb57f3112a740dd294449f"}
Sep 09 12:25:23.243826 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:23.243778    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-fs9wk" podStartSLOduration=1.243763182 podStartE2EDuration="1.243763182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:25:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:25:23.24267723 +0000 UTC m=+342.939242382" watchObservedRunningTime="2026-09-09 12:25:23.243763182 +0000 UTC m=+342.940328333"
Sep 09 12:25:43.753747 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:43.753667    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"]
Sep 09 12:25:43.758282 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:43.758265    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"
Sep 09 12:25:43.760278 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:43.760258    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 12:25:43.760923 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:43.760907    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 12:25:43.761002 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:43.760935    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sghgc\""
Sep 09 12:25:43.764602 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:43.764579    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"]
Sep 09 12:25:43.855153 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:43.855121    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2bqt\" (UniqueName: \"kubernetes.io/projected/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-kube-api-access-b2bqt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74\" (UID: \"7430ea80-3fef-4fd1-bc24-a66dd654e5c4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"
Sep 09 12:25:43.855324 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:43.855174    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74\" (UID: \"7430ea80-3fef-4fd1-bc24-a66dd654e5c4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"
Sep 09 12:25:43.855324 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:43.855262    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74\" (UID: \"7430ea80-3fef-4fd1-bc24-a66dd654e5c4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"
Sep 09 12:25:43.956477 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:43.956442    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bqt\" (UniqueName: \"kubernetes.io/projected/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-kube-api-access-b2bqt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74\" (UID: \"7430ea80-3fef-4fd1-bc24-a66dd654e5c4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"
Sep 09 12:25:43.956716 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:43.956497    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74\" (UID: \"7430ea80-3fef-4fd1-bc24-a66dd654e5c4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"
Sep 09 12:25:43.956716 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:43.956538    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74\" (UID: \"7430ea80-3fef-4fd1-bc24-a66dd654e5c4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"
Sep 09 12:25:43.957004 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:43.956984    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74\" (UID: \"7430ea80-3fef-4fd1-bc24-a66dd654e5c4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"
Sep 09 12:25:43.957080 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:43.957013    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74\" (UID: \"7430ea80-3fef-4fd1-bc24-a66dd654e5c4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"
Sep 09 12:25:43.965110 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:43.965086    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2bqt\" (UniqueName: \"kubernetes.io/projected/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-kube-api-access-b2bqt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74\" (UID: \"7430ea80-3fef-4fd1-bc24-a66dd654e5c4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"
Sep 09 12:25:44.068005 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:44.067925    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"
Sep 09 12:25:44.340806 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:44.340628    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"]
Sep 09 12:25:44.343329 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:25:44.343293    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7430ea80_3fef_4fd1_bc24_a66dd654e5c4.slice/crio-a9d55f9cb3e72373eaf9fee18e116ec74694a5c3f2fc8a1c057bfc3fd296c77e WatchSource:0}: Error finding container a9d55f9cb3e72373eaf9fee18e116ec74694a5c3f2fc8a1c057bfc3fd296c77e: Status 404 returned error can't find the container with id a9d55f9cb3e72373eaf9fee18e116ec74694a5c3f2fc8a1c057bfc3fd296c77e
Sep 09 12:25:45.316819 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:45.316781    2582 generic.go:356] "Generic (PLEG): container finished" podID="7430ea80-3fef-4fd1-bc24-a66dd654e5c4" containerID="6522f8dc0b8b0c99faa29e1655bfda2e961fde677f44c081a09f74056fa0c04a" exitCode=0
Sep 09 12:25:45.317225 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:45.316889    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74" event={"ID":"7430ea80-3fef-4fd1-bc24-a66dd654e5c4","Type":"ContainerDied","Data":"6522f8dc0b8b0c99faa29e1655bfda2e961fde677f44c081a09f74056fa0c04a"}
Sep 09 12:25:45.317225 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:45.316936    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74" event={"ID":"7430ea80-3fef-4fd1-bc24-a66dd654e5c4","Type":"ContainerStarted","Data":"a9d55f9cb3e72373eaf9fee18e116ec74694a5c3f2fc8a1c057bfc3fd296c77e"}
Sep 09 12:25:46.322659 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:46.322558    2582 generic.go:356] "Generic (PLEG): container finished" podID="7430ea80-3fef-4fd1-bc24-a66dd654e5c4" containerID="2c1439bf12282c43cca27734b2aed003968fe7f370e0825efddcd30190ad1ed8" exitCode=0
Sep 09 12:25:46.322659 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:46.322632    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74" event={"ID":"7430ea80-3fef-4fd1-bc24-a66dd654e5c4","Type":"ContainerDied","Data":"2c1439bf12282c43cca27734b2aed003968fe7f370e0825efddcd30190ad1ed8"}
Sep 09 12:25:47.334315 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:47.334278    2582 generic.go:356] "Generic (PLEG): container finished" podID="7430ea80-3fef-4fd1-bc24-a66dd654e5c4" containerID="0d5d4c6d6c1d92f21449989d5f1e5e79ebf8be372967701b43019039c7b09695" exitCode=0
Sep 09 12:25:47.334663 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:47.334344    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74" event={"ID":"7430ea80-3fef-4fd1-bc24-a66dd654e5c4","Type":"ContainerDied","Data":"0d5d4c6d6c1d92f21449989d5f1e5e79ebf8be372967701b43019039c7b09695"}
Sep 09 12:25:48.460792 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:48.460764    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"
Sep 09 12:25:48.592182 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:48.592103    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-bundle\") pod \"7430ea80-3fef-4fd1-bc24-a66dd654e5c4\" (UID: \"7430ea80-3fef-4fd1-bc24-a66dd654e5c4\") "
Sep 09 12:25:48.592182 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:48.592168    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-util\") pod \"7430ea80-3fef-4fd1-bc24-a66dd654e5c4\" (UID: \"7430ea80-3fef-4fd1-bc24-a66dd654e5c4\") "
Sep 09 12:25:48.592362 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:48.592210    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2bqt\" (UniqueName: \"kubernetes.io/projected/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-kube-api-access-b2bqt\") pod \"7430ea80-3fef-4fd1-bc24-a66dd654e5c4\" (UID: \"7430ea80-3fef-4fd1-bc24-a66dd654e5c4\") "
Sep 09 12:25:48.593272 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:48.593239    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-bundle" (OuterVolumeSpecName: "bundle") pod "7430ea80-3fef-4fd1-bc24-a66dd654e5c4" (UID: "7430ea80-3fef-4fd1-bc24-a66dd654e5c4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:25:48.594500 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:48.594469    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-kube-api-access-b2bqt" (OuterVolumeSpecName: "kube-api-access-b2bqt") pod "7430ea80-3fef-4fd1-bc24-a66dd654e5c4" (UID: "7430ea80-3fef-4fd1-bc24-a66dd654e5c4"). InnerVolumeSpecName "kube-api-access-b2bqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:25:48.600483 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:48.600438    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-util" (OuterVolumeSpecName: "util") pod "7430ea80-3fef-4fd1-bc24-a66dd654e5c4" (UID: "7430ea80-3fef-4fd1-bc24-a66dd654e5c4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:25:48.693560 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:48.693527    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-bundle\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:25:48.693560 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:48.693554    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-util\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:25:48.693706 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:48.693567    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2bqt\" (UniqueName: \"kubernetes.io/projected/7430ea80-3fef-4fd1-bc24-a66dd654e5c4-kube-api-access-b2bqt\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:25:49.346153 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:49.346111    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74" event={"ID":"7430ea80-3fef-4fd1-bc24-a66dd654e5c4","Type":"ContainerDied","Data":"a9d55f9cb3e72373eaf9fee18e116ec74694a5c3f2fc8a1c057bfc3fd296c77e"}
Sep 09 12:25:49.346153 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:49.346149    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvrz74"
Sep 09 12:25:49.346399 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:25:49.346172    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9d55f9cb3e72373eaf9fee18e116ec74694a5c3f2fc8a1c057bfc3fd296c77e"
Sep 09 12:26:03.581560 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.581531    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"]
Sep 09 12:26:03.582026 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.581817    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7430ea80-3fef-4fd1-bc24-a66dd654e5c4" containerName="pull"
Sep 09 12:26:03.582026 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.581828    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7430ea80-3fef-4fd1-bc24-a66dd654e5c4" containerName="pull"
Sep 09 12:26:03.582026 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.581843    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7430ea80-3fef-4fd1-bc24-a66dd654e5c4" containerName="extract"
Sep 09 12:26:03.582026 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.581868    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7430ea80-3fef-4fd1-bc24-a66dd654e5c4" containerName="extract"
Sep 09 12:26:03.582026 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.581889    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7430ea80-3fef-4fd1-bc24-a66dd654e5c4" containerName="util"
Sep 09 12:26:03.582026 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.581896    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7430ea80-3fef-4fd1-bc24-a66dd654e5c4" containerName="util"
Sep 09 12:26:03.582026 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.581945    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7430ea80-3fef-4fd1-bc24-a66dd654e5c4" containerName="extract"
Sep 09 12:26:03.585083 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.585062    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"]
Sep 09 12:26:03.585198 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.585179    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.588508 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.587755    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4l7r7\""
Sep 09 12:26:03.588508 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.588089    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 09 12:26:03.588508 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.588243    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 09 12:26:03.588508 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.588332    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 09 12:26:03.588508 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.588458    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 09 12:26:03.591697 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.590801    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"
Sep 09 12:26:03.593288 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.593261    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sghgc\""
Sep 09 12:26:03.593407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.593264    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 12:26:03.593407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.593303    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 12:26:03.600917 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.600896    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"]
Sep 09 12:26:03.602791 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.602771    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"]
Sep 09 12:26:03.699058 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.699027    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9b86bae9-72d0-48c2-b3c2-91651d80a6d3-manifests\") pod \"opendatahub-operator-controller-manager-6c5d49f968-x466m\" (UID: \"9b86bae9-72d0-48c2-b3c2-91651d80a6d3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.699225 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.699067    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b86bae9-72d0-48c2-b3c2-91651d80a6d3-tmp\") pod \"opendatahub-operator-controller-manager-6c5d49f968-x466m\" (UID: \"9b86bae9-72d0-48c2-b3c2-91651d80a6d3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.699225 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.699163    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9b86bae9-72d0-48c2-b3c2-91651d80a6d3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c5d49f968-x466m\" (UID: \"9b86bae9-72d0-48c2-b3c2-91651d80a6d3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.699225 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.699195    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd\" (UID: \"a07f5d24-8138-4ecd-9bc0-bdd3feb45824\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"
Sep 09 12:26:03.699225 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.699217    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-487x2\" (UniqueName: \"kubernetes.io/projected/9b86bae9-72d0-48c2-b3c2-91651d80a6d3-kube-api-access-487x2\") pod \"opendatahub-operator-controller-manager-6c5d49f968-x466m\" (UID: \"9b86bae9-72d0-48c2-b3c2-91651d80a6d3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.699395 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.699303    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b86bae9-72d0-48c2-b3c2-91651d80a6d3-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c5d49f968-x466m\" (UID: \"9b86bae9-72d0-48c2-b3c2-91651d80a6d3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.699484 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.699468    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd\" (UID: \"a07f5d24-8138-4ecd-9bc0-bdd3feb45824\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"
Sep 09 12:26:03.699520 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.699510    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4q2d\" (UniqueName: \"kubernetes.io/projected/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-kube-api-access-g4q2d\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd\" (UID: \"a07f5d24-8138-4ecd-9bc0-bdd3feb45824\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"
Sep 09 12:26:03.800266 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.800232    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-487x2\" (UniqueName: \"kubernetes.io/projected/9b86bae9-72d0-48c2-b3c2-91651d80a6d3-kube-api-access-487x2\") pod \"opendatahub-operator-controller-manager-6c5d49f968-x466m\" (UID: \"9b86bae9-72d0-48c2-b3c2-91651d80a6d3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.800266 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.800270    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b86bae9-72d0-48c2-b3c2-91651d80a6d3-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c5d49f968-x466m\" (UID: \"9b86bae9-72d0-48c2-b3c2-91651d80a6d3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.800471 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.800299    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd\" (UID: \"a07f5d24-8138-4ecd-9bc0-bdd3feb45824\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"
Sep 09 12:26:03.800471 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.800336    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4q2d\" (UniqueName: \"kubernetes.io/projected/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-kube-api-access-g4q2d\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd\" (UID: \"a07f5d24-8138-4ecd-9bc0-bdd3feb45824\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"
Sep 09 12:26:03.800471 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.800357    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9b86bae9-72d0-48c2-b3c2-91651d80a6d3-manifests\") pod \"opendatahub-operator-controller-manager-6c5d49f968-x466m\" (UID: \"9b86bae9-72d0-48c2-b3c2-91651d80a6d3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.800471 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.800375    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b86bae9-72d0-48c2-b3c2-91651d80a6d3-tmp\") pod \"opendatahub-operator-controller-manager-6c5d49f968-x466m\" (UID: \"9b86bae9-72d0-48c2-b3c2-91651d80a6d3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.800471 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.800416    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9b86bae9-72d0-48c2-b3c2-91651d80a6d3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c5d49f968-x466m\" (UID: \"9b86bae9-72d0-48c2-b3c2-91651d80a6d3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.800471 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.800448    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd\" (UID: \"a07f5d24-8138-4ecd-9bc0-bdd3feb45824\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"
Sep 09 12:26:03.800824 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.800798    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9b86bae9-72d0-48c2-b3c2-91651d80a6d3-manifests\") pod \"opendatahub-operator-controller-manager-6c5d49f968-x466m\" (UID: \"9b86bae9-72d0-48c2-b3c2-91651d80a6d3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.800939 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.800840    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd\" (UID: \"a07f5d24-8138-4ecd-9bc0-bdd3feb45824\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"
Sep 09 12:26:03.800939 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.800798    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd\" (UID: \"a07f5d24-8138-4ecd-9bc0-bdd3feb45824\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"
Sep 09 12:26:03.802884 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.802831    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b86bae9-72d0-48c2-b3c2-91651d80a6d3-tmp\") pod \"opendatahub-operator-controller-manager-6c5d49f968-x466m\" (UID: \"9b86bae9-72d0-48c2-b3c2-91651d80a6d3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.803022 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.803006    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9b86bae9-72d0-48c2-b3c2-91651d80a6d3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c5d49f968-x466m\" (UID: \"9b86bae9-72d0-48c2-b3c2-91651d80a6d3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.803086 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.803031    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b86bae9-72d0-48c2-b3c2-91651d80a6d3-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c5d49f968-x466m\" (UID: \"9b86bae9-72d0-48c2-b3c2-91651d80a6d3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.808446 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.808418    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4q2d\" (UniqueName: \"kubernetes.io/projected/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-kube-api-access-g4q2d\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd\" (UID: \"a07f5d24-8138-4ecd-9bc0-bdd3feb45824\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"
Sep 09 12:26:03.808532 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.808454    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-487x2\" (UniqueName: \"kubernetes.io/projected/9b86bae9-72d0-48c2-b3c2-91651d80a6d3-kube-api-access-487x2\") pod \"opendatahub-operator-controller-manager-6c5d49f968-x466m\" (UID: \"9b86bae9-72d0-48c2-b3c2-91651d80a6d3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.901633 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.901547    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:03.908538 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:03.908516    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"
Sep 09 12:26:04.189643 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:04.189618    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"]
Sep 09 12:26:04.191895 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:26:04.191864    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b86bae9_72d0_48c2_b3c2_91651d80a6d3.slice/crio-a6616e3f0a17f2f19318ef8112067de4c2dd060a4a2acacf36d9e1874515356e WatchSource:0}: Error finding container a6616e3f0a17f2f19318ef8112067de4c2dd060a4a2acacf36d9e1874515356e: Status 404 returned error can't find the container with id a6616e3f0a17f2f19318ef8112067de4c2dd060a4a2acacf36d9e1874515356e
Sep 09 12:26:04.197776 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:04.197752    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"]
Sep 09 12:26:04.200565 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:26:04.200542    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda07f5d24_8138_4ecd_9bc0_bdd3feb45824.slice/crio-c5b9c8a36b0ea2b3886ae905c375e800bc7eee59253e8987df905d4d0c26e211 WatchSource:0}: Error finding container c5b9c8a36b0ea2b3886ae905c375e800bc7eee59253e8987df905d4d0c26e211: Status 404 returned error can't find the container with id c5b9c8a36b0ea2b3886ae905c375e800bc7eee59253e8987df905d4d0c26e211
Sep 09 12:26:04.412643 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:04.412615    2582 generic.go:356] "Generic (PLEG): container finished" podID="a07f5d24-8138-4ecd-9bc0-bdd3feb45824" containerID="682c17c016222077433757f725e89043c730e9c21e210437d858edf1bccd87d0" exitCode=0
Sep 09 12:26:04.412802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:04.412706    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd" event={"ID":"a07f5d24-8138-4ecd-9bc0-bdd3feb45824","Type":"ContainerDied","Data":"682c17c016222077433757f725e89043c730e9c21e210437d858edf1bccd87d0"}
Sep 09 12:26:04.412802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:04.412768    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd" event={"ID":"a07f5d24-8138-4ecd-9bc0-bdd3feb45824","Type":"ContainerStarted","Data":"c5b9c8a36b0ea2b3886ae905c375e800bc7eee59253e8987df905d4d0c26e211"}
Sep 09 12:26:04.414282 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:04.414231    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m" event={"ID":"9b86bae9-72d0-48c2-b3c2-91651d80a6d3","Type":"ContainerStarted","Data":"a6616e3f0a17f2f19318ef8112067de4c2dd060a4a2acacf36d9e1874515356e"}
Sep 09 12:26:07.431542 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:07.431509    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m" event={"ID":"9b86bae9-72d0-48c2-b3c2-91651d80a6d3","Type":"ContainerStarted","Data":"c27a9d8b6af8d55fb550e07a95f6655715c30c1ea809ed9a73adbbcdd3d23b84"}
Sep 09 12:26:08.437869 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:08.437820    2582 generic.go:356] "Generic (PLEG): container finished" podID="9b86bae9-72d0-48c2-b3c2-91651d80a6d3" containerID="c27a9d8b6af8d55fb550e07a95f6655715c30c1ea809ed9a73adbbcdd3d23b84" exitCode=0
Sep 09 12:26:08.438258 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:08.437903    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m" event={"ID":"9b86bae9-72d0-48c2-b3c2-91651d80a6d3","Type":"ContainerDied","Data":"c27a9d8b6af8d55fb550e07a95f6655715c30c1ea809ed9a73adbbcdd3d23b84"}
Sep 09 12:26:08.439917 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:08.439896    2582 generic.go:356] "Generic (PLEG): container finished" podID="a07f5d24-8138-4ecd-9bc0-bdd3feb45824" containerID="91127a06e35d0ebd7efb049dea8657bd23c8fc6dbf57ef5873d482ee0f83baed" exitCode=0
Sep 09 12:26:08.440019 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:08.439923    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd" event={"ID":"a07f5d24-8138-4ecd-9bc0-bdd3feb45824","Type":"ContainerDied","Data":"91127a06e35d0ebd7efb049dea8657bd23c8fc6dbf57ef5873d482ee0f83baed"}
Sep 09 12:26:09.446373 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:09.446336    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m" event={"ID":"9b86bae9-72d0-48c2-b3c2-91651d80a6d3","Type":"ContainerStarted","Data":"b6dea7c24dab7d8787d496b3977237e13e1b5657eb0065fc767302df1d1248c6"}
Sep 09 12:26:09.446867 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:09.446423    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:09.448595 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:09.448575    2582 generic.go:356] "Generic (PLEG): container finished" podID="a07f5d24-8138-4ecd-9bc0-bdd3feb45824" containerID="5c16bcf500a5372e3b58d8fd75d10b282db24d5c530d8ed6735e4c63c2771894" exitCode=0
Sep 09 12:26:09.448694 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:09.448620    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd" event={"ID":"a07f5d24-8138-4ecd-9bc0-bdd3feb45824","Type":"ContainerDied","Data":"5c16bcf500a5372e3b58d8fd75d10b282db24d5c530d8ed6735e4c63c2771894"}
Sep 09 12:26:09.471193 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:09.471142    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m" podStartSLOduration=1.870705551 podStartE2EDuration="6.471116677s" totalImagesPullingTime="3.545919565s" totalInitContainerRuntime="1.054491561s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:26:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:26:09.469176147 +0000 UTC m=+389.165741416" watchObservedRunningTime="2026-09-09 12:26:09.471116677 +0000 UTC m=+389.167681830"
Sep 09 12:26:10.577551 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:10.577518    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"
Sep 09 12:26:10.650203 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:10.650167    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-bundle\") pod \"a07f5d24-8138-4ecd-9bc0-bdd3feb45824\" (UID: \"a07f5d24-8138-4ecd-9bc0-bdd3feb45824\") "
Sep 09 12:26:10.650364 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:10.650227    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-util\") pod \"a07f5d24-8138-4ecd-9bc0-bdd3feb45824\" (UID: \"a07f5d24-8138-4ecd-9bc0-bdd3feb45824\") "
Sep 09 12:26:10.650364 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:10.650286    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4q2d\" (UniqueName: \"kubernetes.io/projected/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-kube-api-access-g4q2d\") pod \"a07f5d24-8138-4ecd-9bc0-bdd3feb45824\" (UID: \"a07f5d24-8138-4ecd-9bc0-bdd3feb45824\") "
Sep 09 12:26:10.650897 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:10.650872    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-bundle" (OuterVolumeSpecName: "bundle") pod "a07f5d24-8138-4ecd-9bc0-bdd3feb45824" (UID: "a07f5d24-8138-4ecd-9bc0-bdd3feb45824"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:26:10.652483 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:10.652455    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-kube-api-access-g4q2d" (OuterVolumeSpecName: "kube-api-access-g4q2d") pod "a07f5d24-8138-4ecd-9bc0-bdd3feb45824" (UID: "a07f5d24-8138-4ecd-9bc0-bdd3feb45824"). InnerVolumeSpecName "kube-api-access-g4q2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:26:10.654714 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:10.654694    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-util" (OuterVolumeSpecName: "util") pod "a07f5d24-8138-4ecd-9bc0-bdd3feb45824" (UID: "a07f5d24-8138-4ecd-9bc0-bdd3feb45824"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:26:10.751655 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:10.751623    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4q2d\" (UniqueName: \"kubernetes.io/projected/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-kube-api-access-g4q2d\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:10.751655 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:10.751654    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-bundle\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:10.751828 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:10.751664    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a07f5d24-8138-4ecd-9bc0-bdd3feb45824-util\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:11.458959 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:11.458933    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd"
Sep 09 12:26:11.458959 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:11.458939    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0ndvvd" event={"ID":"a07f5d24-8138-4ecd-9bc0-bdd3feb45824","Type":"ContainerDied","Data":"c5b9c8a36b0ea2b3886ae905c375e800bc7eee59253e8987df905d4d0c26e211"}
Sep 09 12:26:11.459199 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:11.458981    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5b9c8a36b0ea2b3886ae905c375e800bc7eee59253e8987df905d4d0c26e211"
Sep 09 12:26:18.390644 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.390610    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-749d795b56-bp9ph"]
Sep 09 12:26:18.391059 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.390902    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a07f5d24-8138-4ecd-9bc0-bdd3feb45824" containerName="util"
Sep 09 12:26:18.391059 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.390914    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a07f5d24-8138-4ecd-9bc0-bdd3feb45824" containerName="util"
Sep 09 12:26:18.391059 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.390921    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a07f5d24-8138-4ecd-9bc0-bdd3feb45824" containerName="extract"
Sep 09 12:26:18.391059 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.390927    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a07f5d24-8138-4ecd-9bc0-bdd3feb45824" containerName="extract"
Sep 09 12:26:18.391059 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.390935    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a07f5d24-8138-4ecd-9bc0-bdd3feb45824" containerName="pull"
Sep 09 12:26:18.391059 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.390940    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a07f5d24-8138-4ecd-9bc0-bdd3feb45824" containerName="pull"
Sep 09 12:26:18.391059 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.391008    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a07f5d24-8138-4ecd-9bc0-bdd3feb45824" containerName="extract"
Sep 09 12:26:18.397582 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.397562    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.406200 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.406168    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-749d795b56-bp9ph"]
Sep 09 12:26:18.408658 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.408626    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65643f07-410d-45a0-a65a-b0cf8c547397-service-ca\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.408786 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.408674    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65643f07-410d-45a0-a65a-b0cf8c547397-console-serving-cert\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.408786 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.408742    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65643f07-410d-45a0-a65a-b0cf8c547397-console-config\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.408786 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.408778    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7n4j\" (UniqueName: \"kubernetes.io/projected/65643f07-410d-45a0-a65a-b0cf8c547397-kube-api-access-j7n4j\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.408972 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.408801    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65643f07-410d-45a0-a65a-b0cf8c547397-trusted-ca-bundle\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.408972 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.408824    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65643f07-410d-45a0-a65a-b0cf8c547397-console-oauth-config\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.408972 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.408916    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65643f07-410d-45a0-a65a-b0cf8c547397-oauth-serving-cert\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.509918 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.509890    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65643f07-410d-45a0-a65a-b0cf8c547397-trusted-ca-bundle\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.509918 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.509917    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65643f07-410d-45a0-a65a-b0cf8c547397-console-oauth-config\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.510141 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.509960    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65643f07-410d-45a0-a65a-b0cf8c547397-oauth-serving-cert\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.510141 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.510008    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65643f07-410d-45a0-a65a-b0cf8c547397-service-ca\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.510141 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.510035    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65643f07-410d-45a0-a65a-b0cf8c547397-console-serving-cert\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.510141 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.510072    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65643f07-410d-45a0-a65a-b0cf8c547397-console-config\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.510141 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.510099    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7n4j\" (UniqueName: \"kubernetes.io/projected/65643f07-410d-45a0-a65a-b0cf8c547397-kube-api-access-j7n4j\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.510815 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.510791    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65643f07-410d-45a0-a65a-b0cf8c547397-trusted-ca-bundle\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.510963 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.510942    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65643f07-410d-45a0-a65a-b0cf8c547397-console-config\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.511018 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.510974    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65643f07-410d-45a0-a65a-b0cf8c547397-oauth-serving-cert\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.511018 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.510999    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65643f07-410d-45a0-a65a-b0cf8c547397-service-ca\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.512465 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.512446    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65643f07-410d-45a0-a65a-b0cf8c547397-console-oauth-config\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.512564 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.512545    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65643f07-410d-45a0-a65a-b0cf8c547397-console-serving-cert\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.517817 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.517793    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7n4j\" (UniqueName: \"kubernetes.io/projected/65643f07-410d-45a0-a65a-b0cf8c547397-kube-api-access-j7n4j\") pod \"console-749d795b56-bp9ph\" (UID: \"65643f07-410d-45a0-a65a-b0cf8c547397\") " pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.706890 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.706797    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:18.979680 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:18.979592    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-749d795b56-bp9ph"]
Sep 09 12:26:18.982413 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:26:18.982388    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65643f07_410d_45a0_a65a_b0cf8c547397.slice/crio-721da7e59aa45263ea95a07165fb2a05cbf1393d31a97a31ada6f92b60bce7c8 WatchSource:0}: Error finding container 721da7e59aa45263ea95a07165fb2a05cbf1393d31a97a31ada6f92b60bce7c8: Status 404 returned error can't find the container with id 721da7e59aa45263ea95a07165fb2a05cbf1393d31a97a31ada6f92b60bce7c8
Sep 09 12:26:19.495776 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:19.495742    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749d795b56-bp9ph" event={"ID":"65643f07-410d-45a0-a65a-b0cf8c547397","Type":"ContainerStarted","Data":"de0e023ff266e72a7168ac21a63de3613a7d64b6312a5dbb224cce36437b7c85"}
Sep 09 12:26:19.496232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:19.495797    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749d795b56-bp9ph" event={"ID":"65643f07-410d-45a0-a65a-b0cf8c547397","Type":"ContainerStarted","Data":"721da7e59aa45263ea95a07165fb2a05cbf1393d31a97a31ada6f92b60bce7c8"}
Sep 09 12:26:19.514305 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:19.514251    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-749d795b56-bp9ph" podStartSLOduration=1.514235234 podStartE2EDuration="1.514235234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:26:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:26:19.512139541 +0000 UTC m=+399.208704693" watchObservedRunningTime="2026-09-09 12:26:19.514235234 +0000 UTC m=+399.210800387"
Sep 09 12:26:20.455757 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:20.455714    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-x466m"
Sep 09 12:26:25.441013 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.440975    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz"]
Sep 09 12:26:25.444550 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.444529    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz"
Sep 09 12:26:25.447455 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.447421    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 09 12:26:25.447576 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.447491    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-w88fg\""
Sep 09 12:26:25.447908 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.447888    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 09 12:26:25.447908 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.447903    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 09 12:26:25.448069 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.447888    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 09 12:26:25.456148 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.456124    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz"]
Sep 09 12:26:25.468270 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.468249    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6946ad55-ed72-48f5-a38d-a722eac0e7f0-tmp\") pod \"kube-auth-proxy-5948bbb55b-hznrz\" (UID: \"6946ad55-ed72-48f5-a38d-a722eac0e7f0\") " pod="openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz"
Sep 09 12:26:25.468390 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.468358    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cvr8\" (UniqueName: \"kubernetes.io/projected/6946ad55-ed72-48f5-a38d-a722eac0e7f0-kube-api-access-2cvr8\") pod \"kube-auth-proxy-5948bbb55b-hznrz\" (UID: \"6946ad55-ed72-48f5-a38d-a722eac0e7f0\") " pod="openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz"
Sep 09 12:26:25.468457 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.468415    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6946ad55-ed72-48f5-a38d-a722eac0e7f0-tls-certs\") pod \"kube-auth-proxy-5948bbb55b-hznrz\" (UID: \"6946ad55-ed72-48f5-a38d-a722eac0e7f0\") " pod="openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz"
Sep 09 12:26:25.569474 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.569439    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cvr8\" (UniqueName: \"kubernetes.io/projected/6946ad55-ed72-48f5-a38d-a722eac0e7f0-kube-api-access-2cvr8\") pod \"kube-auth-proxy-5948bbb55b-hznrz\" (UID: \"6946ad55-ed72-48f5-a38d-a722eac0e7f0\") " pod="openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz"
Sep 09 12:26:25.569777 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.569741    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6946ad55-ed72-48f5-a38d-a722eac0e7f0-tls-certs\") pod \"kube-auth-proxy-5948bbb55b-hznrz\" (UID: \"6946ad55-ed72-48f5-a38d-a722eac0e7f0\") " pod="openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz"
Sep 09 12:26:25.569914 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.569786    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6946ad55-ed72-48f5-a38d-a722eac0e7f0-tmp\") pod \"kube-auth-proxy-5948bbb55b-hznrz\" (UID: \"6946ad55-ed72-48f5-a38d-a722eac0e7f0\") " pod="openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz"
Sep 09 12:26:25.572267 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.572247    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6946ad55-ed72-48f5-a38d-a722eac0e7f0-tmp\") pod \"kube-auth-proxy-5948bbb55b-hznrz\" (UID: \"6946ad55-ed72-48f5-a38d-a722eac0e7f0\") " pod="openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz"
Sep 09 12:26:25.572509 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.572490    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6946ad55-ed72-48f5-a38d-a722eac0e7f0-tls-certs\") pod \"kube-auth-proxy-5948bbb55b-hznrz\" (UID: \"6946ad55-ed72-48f5-a38d-a722eac0e7f0\") " pod="openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz"
Sep 09 12:26:25.577841 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.577816    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cvr8\" (UniqueName: \"kubernetes.io/projected/6946ad55-ed72-48f5-a38d-a722eac0e7f0-kube-api-access-2cvr8\") pod \"kube-auth-proxy-5948bbb55b-hznrz\" (UID: \"6946ad55-ed72-48f5-a38d-a722eac0e7f0\") " pod="openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz"
Sep 09 12:26:25.755603 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:25.755574    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz"
Sep 09 12:26:26.039514 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.039494    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz"]
Sep 09 12:26:26.041786 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:26:26.041753    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6946ad55_ed72_48f5_a38d_a722eac0e7f0.slice/crio-9ccabc2fa28ef34ba3e3b5f1fed9be4a68479d025c39e0faf26da06951ab4f45 WatchSource:0}: Error finding container 9ccabc2fa28ef34ba3e3b5f1fed9be4a68479d025c39e0faf26da06951ab4f45: Status 404 returned error can't find the container with id 9ccabc2fa28ef34ba3e3b5f1fed9be4a68479d025c39e0faf26da06951ab4f45
Sep 09 12:26:26.528119 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.528087    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz" event={"ID":"6946ad55-ed72-48f5-a38d-a722eac0e7f0","Type":"ContainerStarted","Data":"9ccabc2fa28ef34ba3e3b5f1fed9be4a68479d025c39e0faf26da06951ab4f45"}
Sep 09 12:26:26.829703 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.829624    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"]
Sep 09 12:26:26.832954 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.832932    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.835469 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.835444    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 09 12:26:26.835605 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.835490    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bswxj\""
Sep 09 12:26:26.835674 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.835654    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 09 12:26:26.835733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.835697    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 09 12:26:26.835833 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.835814    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 12:26:26.846521 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.846373    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"]
Sep 09 12:26:26.879518 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.879488    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.879658 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.879537    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.879658 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.879570    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2ppc\" (UniqueName: \"kubernetes.io/projected/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-kube-api-access-t2ppc\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.879658 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.879598    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.879658 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.879637    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.879902 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.879666    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.879902 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.879702    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.981070 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.981038    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.981297 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.981105    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.981297 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.981143    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.981297 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.981171    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2ppc\" (UniqueName: \"kubernetes.io/projected/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-kube-api-access-t2ppc\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.981297 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.981201    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.981297 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.981245    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.981297 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.981274    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.982077 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.982053    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.984748 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.984400    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.984748 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.984708    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.984940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.984774    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.985345 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.985306    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.991985 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.991939    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:26.992275 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:26.992250    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2ppc\" (UniqueName: \"kubernetes.io/projected/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-kube-api-access-t2ppc\") pod \"istiod-openshift-gateway-7b7995d5cd-4r46z\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:27.146813 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:27.146735    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:27.438112 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:27.438079    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"]
Sep 09 12:26:27.440434 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:26:27.440406    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod397cd4a5_7550_4ce6_9083_32aa2d9ead9e.slice/crio-ce243b2e0a88924c09239c79c5f00c2982e5334500933c3545f6e73c5186a5bb WatchSource:0}: Error finding container ce243b2e0a88924c09239c79c5f00c2982e5334500933c3545f6e73c5186a5bb: Status 404 returned error can't find the container with id ce243b2e0a88924c09239c79c5f00c2982e5334500933c3545f6e73c5186a5bb
Sep 09 12:26:27.535931 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:27.535888    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z" event={"ID":"397cd4a5-7550-4ce6-9083-32aa2d9ead9e","Type":"ContainerStarted","Data":"ce243b2e0a88924c09239c79c5f00c2982e5334500933c3545f6e73c5186a5bb"}
Sep 09 12:26:28.707574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:28.707541    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:28.708009 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:28.707604    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:28.712635 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:28.712614    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:29.548601 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:29.548560    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz" event={"ID":"6946ad55-ed72-48f5-a38d-a722eac0e7f0","Type":"ContainerStarted","Data":"42f34b7465eca82f4082db43c98a074336a0769b26bdc4d1edf8d942a1f38ab2"}
Sep 09 12:26:29.553370 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:29.553307    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-749d795b56-bp9ph"
Sep 09 12:26:29.564329 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:29.564287    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5948bbb55b-hznrz" podStartSLOduration=1.650565136 podStartE2EDuration="4.564270818s" totalImagesPullingTime="2.913705682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:26:29.56387907 +0000 UTC m=+409.260444223" watchObservedRunningTime="2026-09-09 12:26:29.564270818 +0000 UTC m=+409.260835971"
Sep 09 12:26:29.613241 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:29.613213    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-556fd77f69-v5z8d"]
Sep 09 12:26:31.025297 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:31.025258    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:26:31.025534 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:31.025326    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:26:31.561042 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:31.561005    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z" event={"ID":"397cd4a5-7550-4ce6-9083-32aa2d9ead9e","Type":"ContainerStarted","Data":"cb31118d61d517c7154a595aad788c826c183431df1f76501a2de468bb0b4d5e"}
Sep 09 12:26:31.561239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:31.561134    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:31.581535 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:31.581489    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z" podStartSLOduration=1.9994913300000001 podStartE2EDuration="5.581474121s" totalImagesPullingTime="3.581982791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:26:31.578974901 +0000 UTC m=+411.275540063" watchObservedRunningTime="2026-09-09 12:26:31.581474121 +0000 UTC m=+411.278039272"
Sep 09 12:26:32.567706 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:32.567678    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:44.997340 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:44.997312    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2"]
Sep 09 12:26:45.002802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.002779    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2"
Sep 09 12:26:45.005053 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.005031    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 09 12:26:45.005159 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.005089    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-rhpds\""
Sep 09 12:26:45.009328 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.009303    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2"]
Sep 09 12:26:45.018614 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.018592    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk"]
Sep 09 12:26:45.022605 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.022239    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk"
Sep 09 12:26:45.024432 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.024402    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 09 12:26:45.024589 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.024566    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6sl5k\""
Sep 09 12:26:45.034622 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.034601    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk"]
Sep 09 12:26:45.131004 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.130972    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d39ec25-d666-4150-b545-d3d6bf8c9417-cert\") pod \"kserve-controller-manager-d6f4b65b8-xk9r2\" (UID: \"7d39ec25-d666-4150-b545-d3d6bf8c9417\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2"
Sep 09 12:26:45.131272 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.131249    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55xj7\" (UniqueName: \"kubernetes.io/projected/37efbe2e-4604-48f8-982f-babd48f9aba3-kube-api-access-55xj7\") pod \"llmisvc-controller-manager-777c6bd7d5-jn9jk\" (UID: \"37efbe2e-4604-48f8-982f-babd48f9aba3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk"
Sep 09 12:26:45.131371 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.131300    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgqtt\" (UniqueName: \"kubernetes.io/projected/7d39ec25-d666-4150-b545-d3d6bf8c9417-kube-api-access-sgqtt\") pod \"kserve-controller-manager-d6f4b65b8-xk9r2\" (UID: \"7d39ec25-d666-4150-b545-d3d6bf8c9417\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2"
Sep 09 12:26:45.131424 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.131366    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37efbe2e-4604-48f8-982f-babd48f9aba3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jn9jk\" (UID: \"37efbe2e-4604-48f8-982f-babd48f9aba3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk"
Sep 09 12:26:45.232458 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.232425    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55xj7\" (UniqueName: \"kubernetes.io/projected/37efbe2e-4604-48f8-982f-babd48f9aba3-kube-api-access-55xj7\") pod \"llmisvc-controller-manager-777c6bd7d5-jn9jk\" (UID: \"37efbe2e-4604-48f8-982f-babd48f9aba3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk"
Sep 09 12:26:45.232632 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.232478    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgqtt\" (UniqueName: \"kubernetes.io/projected/7d39ec25-d666-4150-b545-d3d6bf8c9417-kube-api-access-sgqtt\") pod \"kserve-controller-manager-d6f4b65b8-xk9r2\" (UID: \"7d39ec25-d666-4150-b545-d3d6bf8c9417\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2"
Sep 09 12:26:45.232632 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.232523    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37efbe2e-4604-48f8-982f-babd48f9aba3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jn9jk\" (UID: \"37efbe2e-4604-48f8-982f-babd48f9aba3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk"
Sep 09 12:26:45.232632 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.232575    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d39ec25-d666-4150-b545-d3d6bf8c9417-cert\") pod \"kserve-controller-manager-d6f4b65b8-xk9r2\" (UID: \"7d39ec25-d666-4150-b545-d3d6bf8c9417\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2"
Sep 09 12:26:45.232787 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:26:45.232677    2582 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 09 12:26:45.232787 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:26:45.232682    2582 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 09 12:26:45.232787 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:26:45.232741    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d39ec25-d666-4150-b545-d3d6bf8c9417-cert podName:7d39ec25-d666-4150-b545-d3d6bf8c9417 nodeName:}" failed. No retries permitted until 2026-09-09 12:26:45.732721859 +0000 UTC m=+425.429286996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7d39ec25-d666-4150-b545-d3d6bf8c9417-cert") pod "kserve-controller-manager-d6f4b65b8-xk9r2" (UID: "7d39ec25-d666-4150-b545-d3d6bf8c9417") : secret "kserve-webhook-server-cert" not found
Sep 09 12:26:45.232787 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:26:45.232757    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37efbe2e-4604-48f8-982f-babd48f9aba3-cert podName:37efbe2e-4604-48f8-982f-babd48f9aba3 nodeName:}" failed. No retries permitted until 2026-09-09 12:26:45.732748816 +0000 UTC m=+425.429313946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/37efbe2e-4604-48f8-982f-babd48f9aba3-cert") pod "llmisvc-controller-manager-777c6bd7d5-jn9jk" (UID: "37efbe2e-4604-48f8-982f-babd48f9aba3") : secret "llmisvc-webhook-server-cert" not found
Sep 09 12:26:45.245437 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.245406    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgqtt\" (UniqueName: \"kubernetes.io/projected/7d39ec25-d666-4150-b545-d3d6bf8c9417-kube-api-access-sgqtt\") pod \"kserve-controller-manager-d6f4b65b8-xk9r2\" (UID: \"7d39ec25-d666-4150-b545-d3d6bf8c9417\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2"
Sep 09 12:26:45.249022 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.248953    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55xj7\" (UniqueName: \"kubernetes.io/projected/37efbe2e-4604-48f8-982f-babd48f9aba3-kube-api-access-55xj7\") pod \"llmisvc-controller-manager-777c6bd7d5-jn9jk\" (UID: \"37efbe2e-4604-48f8-982f-babd48f9aba3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk"
Sep 09 12:26:45.737504 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.737415    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37efbe2e-4604-48f8-982f-babd48f9aba3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jn9jk\" (UID: \"37efbe2e-4604-48f8-982f-babd48f9aba3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk"
Sep 09 12:26:45.737504 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.737472    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d39ec25-d666-4150-b545-d3d6bf8c9417-cert\") pod \"kserve-controller-manager-d6f4b65b8-xk9r2\" (UID: \"7d39ec25-d666-4150-b545-d3d6bf8c9417\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2"
Sep 09 12:26:45.739950 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.739925    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d39ec25-d666-4150-b545-d3d6bf8c9417-cert\") pod \"kserve-controller-manager-d6f4b65b8-xk9r2\" (UID: \"7d39ec25-d666-4150-b545-d3d6bf8c9417\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2"
Sep 09 12:26:45.740058 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.739974    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37efbe2e-4604-48f8-982f-babd48f9aba3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jn9jk\" (UID: \"37efbe2e-4604-48f8-982f-babd48f9aba3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk"
Sep 09 12:26:45.916480 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.916456    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2"
Sep 09 12:26:45.936753 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:45.936730    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk"
Sep 09 12:26:46.192595 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:46.192539    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2"]
Sep 09 12:26:46.194844 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:26:46.194817    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d39ec25_d666_4150_b545_d3d6bf8c9417.slice/crio-8ebab80eaf3bbc168d7579fb5c0af721e41b3854b36c1017b56fb2b56edc4902 WatchSource:0}: Error finding container 8ebab80eaf3bbc168d7579fb5c0af721e41b3854b36c1017b56fb2b56edc4902: Status 404 returned error can't find the container with id 8ebab80eaf3bbc168d7579fb5c0af721e41b3854b36c1017b56fb2b56edc4902
Sep 09 12:26:46.238045 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:46.238022    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk"]
Sep 09 12:26:46.239366 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:26:46.239341    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod37efbe2e_4604_48f8_982f_babd48f9aba3.slice/crio-1fd8f783848625c2122db56bdfad6cd1b458976b82b5e2efe1f54d93e4a6677f WatchSource:0}: Error finding container 1fd8f783848625c2122db56bdfad6cd1b458976b82b5e2efe1f54d93e4a6677f: Status 404 returned error can't find the container with id 1fd8f783848625c2122db56bdfad6cd1b458976b82b5e2efe1f54d93e4a6677f
Sep 09 12:26:46.634180 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:46.634141    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk" event={"ID":"37efbe2e-4604-48f8-982f-babd48f9aba3","Type":"ContainerStarted","Data":"1fd8f783848625c2122db56bdfad6cd1b458976b82b5e2efe1f54d93e4a6677f"}
Sep 09 12:26:46.635394 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:46.635369    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2" event={"ID":"7d39ec25-d666-4150-b545-d3d6bf8c9417","Type":"ContainerStarted","Data":"8ebab80eaf3bbc168d7579fb5c0af721e41b3854b36c1017b56fb2b56edc4902"}
Sep 09 12:26:46.727349 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:46.727319    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"]
Sep 09 12:26:46.727508 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:46.727489    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z" podUID="397cd4a5-7550-4ce6-9083-32aa2d9ead9e" containerName="discovery" containerID="cri-o://cb31118d61d517c7154a595aad788c826c183431df1f76501a2de468bb0b4d5e" gracePeriod=30
Sep 09 12:26:46.993201 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:46.993178    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:47.048740 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.048714    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-csr-dns-cert\") pod \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") "
Sep 09 12:26:47.048915 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.048782    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-cacerts\") pod \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") "
Sep 09 12:26:47.048915 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.048833    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-csr-ca-configmap\") pod \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") "
Sep 09 12:26:47.048915 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.048886    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-token\") pod \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") "
Sep 09 12:26:47.048915 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.048913    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2ppc\" (UniqueName: \"kubernetes.io/projected/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-kube-api-access-t2ppc\") pod \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") "
Sep 09 12:26:47.049161 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.048942    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-local-certs\") pod \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") "
Sep 09 12:26:47.049161 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.048975    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-kubeconfig\") pod \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\" (UID: \"397cd4a5-7550-4ce6-9083-32aa2d9ead9e\") "
Sep 09 12:26:47.049301 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.049272    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "397cd4a5-7550-4ce6-9083-32aa2d9ead9e" (UID: "397cd4a5-7550-4ce6-9083-32aa2d9ead9e"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:26:47.052556 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.052486    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "397cd4a5-7550-4ce6-9083-32aa2d9ead9e" (UID: "397cd4a5-7550-4ce6-9083-32aa2d9ead9e"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:26:47.052556 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.052503    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-kube-api-access-t2ppc" (OuterVolumeSpecName: "kube-api-access-t2ppc") pod "397cd4a5-7550-4ce6-9083-32aa2d9ead9e" (UID: "397cd4a5-7550-4ce6-9083-32aa2d9ead9e"). InnerVolumeSpecName "kube-api-access-t2ppc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:26:47.052556 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.052517    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-token" (OuterVolumeSpecName: "istio-token") pod "397cd4a5-7550-4ce6-9083-32aa2d9ead9e" (UID: "397cd4a5-7550-4ce6-9083-32aa2d9ead9e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:26:47.052798 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.052603    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-cacerts" (OuterVolumeSpecName: "cacerts") pod "397cd4a5-7550-4ce6-9083-32aa2d9ead9e" (UID: "397cd4a5-7550-4ce6-9083-32aa2d9ead9e"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:26:47.052798 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.052652    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-local-certs" (OuterVolumeSpecName: "local-certs") pod "397cd4a5-7550-4ce6-9083-32aa2d9ead9e" (UID: "397cd4a5-7550-4ce6-9083-32aa2d9ead9e"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:26:47.053005 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.052977    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "397cd4a5-7550-4ce6-9083-32aa2d9ead9e" (UID: "397cd4a5-7550-4ce6-9083-32aa2d9ead9e"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:26:47.150733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.150661    2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-csr-ca-configmap\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:47.150733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.150695    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:47.150733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.150710    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2ppc\" (UniqueName: \"kubernetes.io/projected/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-kube-api-access-t2ppc\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:47.150733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.150723    2582 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-local-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:47.150992 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.150737    2582 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-kubeconfig\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:47.150992 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.150750    2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-istio-csr-dns-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:47.150992 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.150763    2582 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/397cd4a5-7550-4ce6-9083-32aa2d9ead9e-cacerts\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:47.642686 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.642651    2582 generic.go:356] "Generic (PLEG): container finished" podID="397cd4a5-7550-4ce6-9083-32aa2d9ead9e" containerID="cb31118d61d517c7154a595aad788c826c183431df1f76501a2de468bb0b4d5e" exitCode=0
Sep 09 12:26:47.643155 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.642727    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"
Sep 09 12:26:47.643155 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.642737    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z" event={"ID":"397cd4a5-7550-4ce6-9083-32aa2d9ead9e","Type":"ContainerDied","Data":"cb31118d61d517c7154a595aad788c826c183431df1f76501a2de468bb0b4d5e"}
Sep 09 12:26:47.643155 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.642801    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z" event={"ID":"397cd4a5-7550-4ce6-9083-32aa2d9ead9e","Type":"ContainerDied","Data":"ce243b2e0a88924c09239c79c5f00c2982e5334500933c3545f6e73c5186a5bb"}
Sep 09 12:26:47.643155 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.642832    2582 scope.go:117] "RemoveContainer" containerID="cb31118d61d517c7154a595aad788c826c183431df1f76501a2de468bb0b4d5e"
Sep 09 12:26:47.666957 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.666932    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"]
Sep 09 12:26:47.671705 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:47.671681    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4r46z"]
Sep 09 12:26:48.409647 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:48.409526    2582 scope.go:117] "RemoveContainer" containerID="cb31118d61d517c7154a595aad788c826c183431df1f76501a2de468bb0b4d5e"
Sep 09 12:26:48.409968 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:26:48.409938    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb31118d61d517c7154a595aad788c826c183431df1f76501a2de468bb0b4d5e\": container with ID starting with cb31118d61d517c7154a595aad788c826c183431df1f76501a2de468bb0b4d5e not found: ID does not exist" containerID="cb31118d61d517c7154a595aad788c826c183431df1f76501a2de468bb0b4d5e"
Sep 09 12:26:48.410043 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:48.409978    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb31118d61d517c7154a595aad788c826c183431df1f76501a2de468bb0b4d5e"} err="failed to get container status \"cb31118d61d517c7154a595aad788c826c183431df1f76501a2de468bb0b4d5e\": rpc error: code = NotFound desc = could not find container \"cb31118d61d517c7154a595aad788c826c183431df1f76501a2de468bb0b4d5e\": container with ID starting with cb31118d61d517c7154a595aad788c826c183431df1f76501a2de468bb0b4d5e not found: ID does not exist"
Sep 09 12:26:48.651417 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:48.651373    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2" event={"ID":"7d39ec25-d666-4150-b545-d3d6bf8c9417","Type":"ContainerStarted","Data":"9b0dd4b2fa1814fcfce3dd148911f968cb7cc7dd1ee5056be65510f408cf2998"}
Sep 09 12:26:48.651838 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:48.651497    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2"
Sep 09 12:26:48.668786 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:48.668696    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2" podStartSLOduration=2.325802134 podStartE2EDuration="4.668681301s" totalImagesPullingTime="2.342879167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:26:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:26:48.666823412 +0000 UTC m=+428.363388565" watchObservedRunningTime="2026-09-09 12:26:48.668681301 +0000 UTC m=+428.365246455"
Sep 09 12:26:48.826396 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:48.826359    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="397cd4a5-7550-4ce6-9083-32aa2d9ead9e" path="/var/lib/kubelet/pods/397cd4a5-7550-4ce6-9083-32aa2d9ead9e/volumes"
Sep 09 12:26:49.661792 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:49.661760    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk" event={"ID":"37efbe2e-4604-48f8-982f-babd48f9aba3","Type":"ContainerStarted","Data":"dd13b37571b26bff7ec83a336bbfa55490a5a911804054be4d6293467165d81c"}
Sep 09 12:26:49.662248 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:49.662042    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk"
Sep 09 12:26:49.681712 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:49.681667    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk" podStartSLOduration=2.684563 podStartE2EDuration="5.681651239s" totalImagesPullingTime="2.997088239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:26:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:26:49.679906675 +0000 UTC m=+429.376471827" watchObservedRunningTime="2026-09-09 12:26:49.681651239 +0000 UTC m=+429.378216391"
Sep 09 12:26:54.638983 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:54.638940    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-556fd77f69-v5z8d" podUID="6e744d90-e9d2-4ac9-b9dd-936a162ac888" containerName="console" containerID="cri-o://2b9371b1d76f680584e3c4071042c4c4f32ffea5cf1d540fc8dd6b104a45e047" gracePeriod=15
Sep 09 12:26:54.883067 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:54.883038    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:26:55.016705 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.016670    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-oauth-serving-cert\") pod \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") "
Sep 09 12:26:55.016897 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.016743    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-oauth-config\") pod \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") "
Sep 09 12:26:55.016897 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.016799    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-trusted-ca-bundle\") pod \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") "
Sep 09 12:26:55.016897 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.016831    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-serving-cert\") pod \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") "
Sep 09 12:26:55.016897 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.016877    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-config\") pod \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") "
Sep 09 12:26:55.017130 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.016917    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59f89\" (UniqueName: \"kubernetes.io/projected/6e744d90-e9d2-4ac9-b9dd-936a162ac888-kube-api-access-59f89\") pod \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") "
Sep 09 12:26:55.017130 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.016946    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-service-ca\") pod \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\" (UID: \"6e744d90-e9d2-4ac9-b9dd-936a162ac888\") "
Sep 09 12:26:55.017253 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.017214    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6e744d90-e9d2-4ac9-b9dd-936a162ac888" (UID: "6e744d90-e9d2-4ac9-b9dd-936a162ac888"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:26:55.017401 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.017293    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6e744d90-e9d2-4ac9-b9dd-936a162ac888" (UID: "6e744d90-e9d2-4ac9-b9dd-936a162ac888"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:26:55.017468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.017453    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-service-ca" (OuterVolumeSpecName: "service-ca") pod "6e744d90-e9d2-4ac9-b9dd-936a162ac888" (UID: "6e744d90-e9d2-4ac9-b9dd-936a162ac888"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:26:55.017549 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.017525    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-config" (OuterVolumeSpecName: "console-config") pod "6e744d90-e9d2-4ac9-b9dd-936a162ac888" (UID: "6e744d90-e9d2-4ac9-b9dd-936a162ac888"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:26:55.019241 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.019208    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6e744d90-e9d2-4ac9-b9dd-936a162ac888" (UID: "6e744d90-e9d2-4ac9-b9dd-936a162ac888"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:26:55.019447 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.019428    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e744d90-e9d2-4ac9-b9dd-936a162ac888-kube-api-access-59f89" (OuterVolumeSpecName: "kube-api-access-59f89") pod "6e744d90-e9d2-4ac9-b9dd-936a162ac888" (UID: "6e744d90-e9d2-4ac9-b9dd-936a162ac888"). InnerVolumeSpecName "kube-api-access-59f89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:26:55.019547 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.019524    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6e744d90-e9d2-4ac9-b9dd-936a162ac888" (UID: "6e744d90-e9d2-4ac9-b9dd-936a162ac888"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:26:55.118296 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.118262    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-oauth-serving-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:55.118410 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.118299    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-oauth-config\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:55.118410 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.118313    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-trusted-ca-bundle\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:55.118410 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.118326    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-serving-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:55.118410 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.118338    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-console-config\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:55.118410 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.118350    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59f89\" (UniqueName: \"kubernetes.io/projected/6e744d90-e9d2-4ac9-b9dd-936a162ac888-kube-api-access-59f89\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:55.118410 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.118361    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e744d90-e9d2-4ac9-b9dd-936a162ac888-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:26:55.698754 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.698717    2582 generic.go:356] "Generic (PLEG): container finished" podID="6e744d90-e9d2-4ac9-b9dd-936a162ac888" containerID="2b9371b1d76f680584e3c4071042c4c4f32ffea5cf1d540fc8dd6b104a45e047" exitCode=0
Sep 09 12:26:55.699240 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.698759    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556fd77f69-v5z8d"
Sep 09 12:26:55.699240 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.698776    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556fd77f69-v5z8d" event={"ID":"6e744d90-e9d2-4ac9-b9dd-936a162ac888","Type":"ContainerDied","Data":"2b9371b1d76f680584e3c4071042c4c4f32ffea5cf1d540fc8dd6b104a45e047"}
Sep 09 12:26:55.699240 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.698829    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556fd77f69-v5z8d" event={"ID":"6e744d90-e9d2-4ac9-b9dd-936a162ac888","Type":"ContainerDied","Data":"7ce0ed7eac52c9d88371825e236b55c2d88db38bef1a2662b516a02b70c5ba25"}
Sep 09 12:26:55.699240 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.698880    2582 scope.go:117] "RemoveContainer" containerID="2b9371b1d76f680584e3c4071042c4c4f32ffea5cf1d540fc8dd6b104a45e047"
Sep 09 12:26:55.707835 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.707819    2582 scope.go:117] "RemoveContainer" containerID="2b9371b1d76f680584e3c4071042c4c4f32ffea5cf1d540fc8dd6b104a45e047"
Sep 09 12:26:55.708141 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:26:55.708123    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b9371b1d76f680584e3c4071042c4c4f32ffea5cf1d540fc8dd6b104a45e047\": container with ID starting with 2b9371b1d76f680584e3c4071042c4c4f32ffea5cf1d540fc8dd6b104a45e047 not found: ID does not exist" containerID="2b9371b1d76f680584e3c4071042c4c4f32ffea5cf1d540fc8dd6b104a45e047"
Sep 09 12:26:55.708187 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.708148    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b9371b1d76f680584e3c4071042c4c4f32ffea5cf1d540fc8dd6b104a45e047"} err="failed to get container status \"2b9371b1d76f680584e3c4071042c4c4f32ffea5cf1d540fc8dd6b104a45e047\": rpc error: code = NotFound desc = could not find container \"2b9371b1d76f680584e3c4071042c4c4f32ffea5cf1d540fc8dd6b104a45e047\": container with ID starting with 2b9371b1d76f680584e3c4071042c4c4f32ffea5cf1d540fc8dd6b104a45e047 not found: ID does not exist"
Sep 09 12:26:55.718684 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.718660    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-556fd77f69-v5z8d"]
Sep 09 12:26:55.720952 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:55.720934    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-556fd77f69-v5z8d"]
Sep 09 12:26:56.824621 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:26:56.824592    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e744d90-e9d2-4ac9-b9dd-936a162ac888" path="/var/lib/kubelet/pods/6e744d90-e9d2-4ac9-b9dd-936a162ac888/volumes"
Sep 09 12:27:19.666936 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:19.666902    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xk9r2"
Sep 09 12:27:20.669521 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:20.669493    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jn9jk"
Sep 09 12:27:41.048536 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.048475    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"]
Sep 09 12:27:41.048888 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.048758    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="397cd4a5-7550-4ce6-9083-32aa2d9ead9e" containerName="discovery"
Sep 09 12:27:41.048888 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.048768    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="397cd4a5-7550-4ce6-9083-32aa2d9ead9e" containerName="discovery"
Sep 09 12:27:41.048888 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.048792    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e744d90-e9d2-4ac9-b9dd-936a162ac888" containerName="console"
Sep 09 12:27:41.048888 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.048798    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e744d90-e9d2-4ac9-b9dd-936a162ac888" containerName="console"
Sep 09 12:27:41.048888 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.048844    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e744d90-e9d2-4ac9-b9dd-936a162ac888" containerName="console"
Sep 09 12:27:41.048888 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.048866    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="397cd4a5-7550-4ce6-9083-32aa2d9ead9e" containerName="discovery"
Sep 09 12:27:41.053038 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.053023    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"
Sep 09 12:27:41.055071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.055046    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sghgc\""
Sep 09 12:27:41.055071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.055059    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 12:27:41.055231 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.055073    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 12:27:41.067355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.067332    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"]
Sep 09 12:27:41.145422 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.145384    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"]
Sep 09 12:27:41.149183 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.149163    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"
Sep 09 12:27:41.156024 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.155997    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"]
Sep 09 12:27:41.166354 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.166329    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7mps\" (UniqueName: \"kubernetes.io/projected/f35af65c-b570-4aa8-80a2-d11558f3c0e1-kube-api-access-q7mps\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5\" (UID: \"f35af65c-b570-4aa8-80a2-d11558f3c0e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"
Sep 09 12:27:41.166476 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.166376    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f35af65c-b570-4aa8-80a2-d11558f3c0e1-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5\" (UID: \"f35af65c-b570-4aa8-80a2-d11558f3c0e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"
Sep 09 12:27:41.166476 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.166432    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f35af65c-b570-4aa8-80a2-d11558f3c0e1-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5\" (UID: \"f35af65c-b570-4aa8-80a2-d11558f3c0e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"
Sep 09 12:27:41.247588 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.247559    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"]
Sep 09 12:27:41.250927 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.250913    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"
Sep 09 12:27:41.259265 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.259237    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"]
Sep 09 12:27:41.266917 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.266897    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6\" (UID: \"598d3ae5-2bec-49bb-8ed1-4a1a20454acb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"
Sep 09 12:27:41.267019 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.266927    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knkgt\" (UniqueName: \"kubernetes.io/projected/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-kube-api-access-knkgt\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6\" (UID: \"598d3ae5-2bec-49bb-8ed1-4a1a20454acb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"
Sep 09 12:27:41.267019 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.266955    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f35af65c-b570-4aa8-80a2-d11558f3c0e1-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5\" (UID: \"f35af65c-b570-4aa8-80a2-d11558f3c0e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"
Sep 09 12:27:41.267019 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.266982    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f35af65c-b570-4aa8-80a2-d11558f3c0e1-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5\" (UID: \"f35af65c-b570-4aa8-80a2-d11558f3c0e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"
Sep 09 12:27:41.267192 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.267025    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7mps\" (UniqueName: \"kubernetes.io/projected/f35af65c-b570-4aa8-80a2-d11558f3c0e1-kube-api-access-q7mps\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5\" (UID: \"f35af65c-b570-4aa8-80a2-d11558f3c0e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"
Sep 09 12:27:41.267192 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.267048    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6\" (UID: \"598d3ae5-2bec-49bb-8ed1-4a1a20454acb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"
Sep 09 12:27:41.267346 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.267325    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f35af65c-b570-4aa8-80a2-d11558f3c0e1-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5\" (UID: \"f35af65c-b570-4aa8-80a2-d11558f3c0e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"
Sep 09 12:27:41.267398 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.267356    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f35af65c-b570-4aa8-80a2-d11558f3c0e1-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5\" (UID: \"f35af65c-b570-4aa8-80a2-d11558f3c0e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"
Sep 09 12:27:41.275016 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.274994    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7mps\" (UniqueName: \"kubernetes.io/projected/f35af65c-b570-4aa8-80a2-d11558f3c0e1-kube-api-access-q7mps\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5\" (UID: \"f35af65c-b570-4aa8-80a2-d11558f3c0e1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"
Sep 09 12:27:41.342795 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.342736    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"]
Sep 09 12:27:41.346230 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.346212    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"
Sep 09 12:27:41.354689 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.354656    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"]
Sep 09 12:27:41.367401 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.367373    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7qj8\" (UniqueName: \"kubernetes.io/projected/4142cdaf-6fcf-4c64-bd49-e0816576268d-kube-api-access-j7qj8\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7\" (UID: \"4142cdaf-6fcf-4c64-bd49-e0816576268d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"
Sep 09 12:27:41.367483 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.367406    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4142cdaf-6fcf-4c64-bd49-e0816576268d-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7\" (UID: \"4142cdaf-6fcf-4c64-bd49-e0816576268d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"
Sep 09 12:27:41.367483 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.367429    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4142cdaf-6fcf-4c64-bd49-e0816576268d-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7\" (UID: \"4142cdaf-6fcf-4c64-bd49-e0816576268d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"
Sep 09 12:27:41.367558 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.367478    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6\" (UID: \"598d3ae5-2bec-49bb-8ed1-4a1a20454acb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"
Sep 09 12:27:41.367558 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.367516    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6\" (UID: \"598d3ae5-2bec-49bb-8ed1-4a1a20454acb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"
Sep 09 12:27:41.367558 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.367545    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knkgt\" (UniqueName: \"kubernetes.io/projected/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-kube-api-access-knkgt\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6\" (UID: \"598d3ae5-2bec-49bb-8ed1-4a1a20454acb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"
Sep 09 12:27:41.367787 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.367771    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"
Sep 09 12:27:41.367874 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.367838    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6\" (UID: \"598d3ae5-2bec-49bb-8ed1-4a1a20454acb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"
Sep 09 12:27:41.367913 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.367876    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6\" (UID: \"598d3ae5-2bec-49bb-8ed1-4a1a20454acb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"
Sep 09 12:27:41.375675 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.375655    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knkgt\" (UniqueName: \"kubernetes.io/projected/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-kube-api-access-knkgt\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6\" (UID: \"598d3ae5-2bec-49bb-8ed1-4a1a20454acb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"
Sep 09 12:27:41.459364 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.459338    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"
Sep 09 12:27:41.467965 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.467942    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4142cdaf-6fcf-4c64-bd49-e0816576268d-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7\" (UID: \"4142cdaf-6fcf-4c64-bd49-e0816576268d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"
Sep 09 12:27:41.468057 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.467983    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4142cdaf-6fcf-4c64-bd49-e0816576268d-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7\" (UID: \"4142cdaf-6fcf-4c64-bd49-e0816576268d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"
Sep 09 12:27:41.468057 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.468026    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj\" (UID: \"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"
Sep 09 12:27:41.468151 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.468058    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj\" (UID: \"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"
Sep 09 12:27:41.468218 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.468198    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7qj8\" (UniqueName: \"kubernetes.io/projected/4142cdaf-6fcf-4c64-bd49-e0816576268d-kube-api-access-j7qj8\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7\" (UID: \"4142cdaf-6fcf-4c64-bd49-e0816576268d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"
Sep 09 12:27:41.468273 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.468241    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kctc2\" (UniqueName: \"kubernetes.io/projected/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-kube-api-access-kctc2\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj\" (UID: \"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"
Sep 09 12:27:41.468320 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.468295    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4142cdaf-6fcf-4c64-bd49-e0816576268d-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7\" (UID: \"4142cdaf-6fcf-4c64-bd49-e0816576268d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"
Sep 09 12:27:41.468395 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.468375    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4142cdaf-6fcf-4c64-bd49-e0816576268d-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7\" (UID: \"4142cdaf-6fcf-4c64-bd49-e0816576268d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"
Sep 09 12:27:41.476930 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.476906    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7qj8\" (UniqueName: \"kubernetes.io/projected/4142cdaf-6fcf-4c64-bd49-e0816576268d-kube-api-access-j7qj8\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7\" (UID: \"4142cdaf-6fcf-4c64-bd49-e0816576268d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"
Sep 09 12:27:41.560347 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.560312    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"
Sep 09 12:27:41.569663 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.569637    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj\" (UID: \"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"
Sep 09 12:27:41.569745 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.569676    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj\" (UID: \"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"
Sep 09 12:27:41.569800 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.569745    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kctc2\" (UniqueName: \"kubernetes.io/projected/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-kube-api-access-kctc2\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj\" (UID: \"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"
Sep 09 12:27:41.570065 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.570044    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj\" (UID: \"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"
Sep 09 12:27:41.570142 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.570106    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj\" (UID: \"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"
Sep 09 12:27:41.578392 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.578367    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kctc2\" (UniqueName: \"kubernetes.io/projected/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-kube-api-access-kctc2\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj\" (UID: \"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"
Sep 09 12:27:41.643031 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.642869    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"]
Sep 09 12:27:41.645569 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:27:41.645541    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf35af65c_b570_4aa8_80a2_d11558f3c0e1.slice/crio-e8cd55b5f090a3c44c30507ff8e6c8fd0ed66eb89806428868ecaface47846bd WatchSource:0}: Error finding container e8cd55b5f090a3c44c30507ff8e6c8fd0ed66eb89806428868ecaface47846bd: Status 404 returned error can't find the container with id e8cd55b5f090a3c44c30507ff8e6c8fd0ed66eb89806428868ecaface47846bd
Sep 09 12:27:41.655086 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.655066    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"
Sep 09 12:27:41.734419 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.734390    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"]
Sep 09 12:27:41.736349 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:27:41.736294    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598d3ae5_2bec_49bb_8ed1_4a1a20454acb.slice/crio-9f60e884f302460b282d1a95de2f354b41caf69ee64512475ce7057ee1a5306c WatchSource:0}: Error finding container 9f60e884f302460b282d1a95de2f354b41caf69ee64512475ce7057ee1a5306c: Status 404 returned error can't find the container with id 9f60e884f302460b282d1a95de2f354b41caf69ee64512475ce7057ee1a5306c
Sep 09 12:27:41.841960 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.841925    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"]
Sep 09 12:27:41.844983 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:27:41.844960    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4142cdaf_6fcf_4c64_bd49_e0816576268d.slice/crio-5c2e214a14fd8402253a379fcdb98b3f7c5e8176ee0327866f99f0387af01a86 WatchSource:0}: Error finding container 5c2e214a14fd8402253a379fcdb98b3f7c5e8176ee0327866f99f0387af01a86: Status 404 returned error can't find the container with id 5c2e214a14fd8402253a379fcdb98b3f7c5e8176ee0327866f99f0387af01a86
Sep 09 12:27:41.925364 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.925336    2582 generic.go:356] "Generic (PLEG): container finished" podID="598d3ae5-2bec-49bb-8ed1-4a1a20454acb" containerID="7175526cc338ec5094a3332d5b06209c23e50144e1f056fbf5b53fde0ed850e6" exitCode=0
Sep 09 12:27:41.925474 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.925411    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6" event={"ID":"598d3ae5-2bec-49bb-8ed1-4a1a20454acb","Type":"ContainerDied","Data":"7175526cc338ec5094a3332d5b06209c23e50144e1f056fbf5b53fde0ed850e6"}
Sep 09 12:27:41.925474 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.925461    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6" event={"ID":"598d3ae5-2bec-49bb-8ed1-4a1a20454acb","Type":"ContainerStarted","Data":"9f60e884f302460b282d1a95de2f354b41caf69ee64512475ce7057ee1a5306c"}
Sep 09 12:27:41.927208 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.927185    2582 generic.go:356] "Generic (PLEG): container finished" podID="f35af65c-b570-4aa8-80a2-d11558f3c0e1" containerID="f357226020f832d5d69b21fd7a3e78c5d2053185c562a1abee94336560497fa0" exitCode=0
Sep 09 12:27:41.927313 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.927261    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5" event={"ID":"f35af65c-b570-4aa8-80a2-d11558f3c0e1","Type":"ContainerDied","Data":"f357226020f832d5d69b21fd7a3e78c5d2053185c562a1abee94336560497fa0"}
Sep 09 12:27:41.927313 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.927301    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5" event={"ID":"f35af65c-b570-4aa8-80a2-d11558f3c0e1","Type":"ContainerStarted","Data":"e8cd55b5f090a3c44c30507ff8e6c8fd0ed66eb89806428868ecaface47846bd"}
Sep 09 12:27:41.928496 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.928472    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7" event={"ID":"4142cdaf-6fcf-4c64-bd49-e0816576268d","Type":"ContainerStarted","Data":"5c2e214a14fd8402253a379fcdb98b3f7c5e8176ee0327866f99f0387af01a86"}
Sep 09 12:27:41.932186 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:41.932169    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"]
Sep 09 12:27:41.934417 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:27:41.934394    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd76b69b1_1ab2_4bad_acc3_eaf81d760cbb.slice/crio-f4e65f42353e3cd9ec9aba2d761caed4e42f795e87f360fdc94392a0d6259ee8 WatchSource:0}: Error finding container f4e65f42353e3cd9ec9aba2d761caed4e42f795e87f360fdc94392a0d6259ee8: Status 404 returned error can't find the container with id f4e65f42353e3cd9ec9aba2d761caed4e42f795e87f360fdc94392a0d6259ee8
Sep 09 12:27:42.936218 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:42.936189    2582 generic.go:356] "Generic (PLEG): container finished" podID="f35af65c-b570-4aa8-80a2-d11558f3c0e1" containerID="938fda81a000fe005296a39a0c29dd8621d8605387e7546051ec192067d68cbd" exitCode=0
Sep 09 12:27:42.936555 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:42.936262    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5" event={"ID":"f35af65c-b570-4aa8-80a2-d11558f3c0e1","Type":"ContainerDied","Data":"938fda81a000fe005296a39a0c29dd8621d8605387e7546051ec192067d68cbd"}
Sep 09 12:27:42.937903 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:42.937817    2582 generic.go:356] "Generic (PLEG): container finished" podID="4142cdaf-6fcf-4c64-bd49-e0816576268d" containerID="6ecd422614e0d30f5d8f295ae185296ce183e7b7757e14a2fe2a7c8d1cc9a70b" exitCode=0
Sep 09 12:27:42.937903 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:42.937881    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7" event={"ID":"4142cdaf-6fcf-4c64-bd49-e0816576268d","Type":"ContainerDied","Data":"6ecd422614e0d30f5d8f295ae185296ce183e7b7757e14a2fe2a7c8d1cc9a70b"}
Sep 09 12:27:42.939474 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:42.939453    2582 generic.go:356] "Generic (PLEG): container finished" podID="d76b69b1-1ab2-4bad-acc3-eaf81d760cbb" containerID="6e228608872c78a36eefbda84b228555a483d6712583d59da866e43fb433ef5a" exitCode=0
Sep 09 12:27:42.939562 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:42.939481    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj" event={"ID":"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb","Type":"ContainerDied","Data":"6e228608872c78a36eefbda84b228555a483d6712583d59da866e43fb433ef5a"}
Sep 09 12:27:42.939562 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:42.939515    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj" event={"ID":"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb","Type":"ContainerStarted","Data":"f4e65f42353e3cd9ec9aba2d761caed4e42f795e87f360fdc94392a0d6259ee8"}
Sep 09 12:27:42.941566 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:42.941543    2582 generic.go:356] "Generic (PLEG): container finished" podID="598d3ae5-2bec-49bb-8ed1-4a1a20454acb" containerID="b48e2fb0d38e959df08294085ac5bc9dc46f4ce520232d5f4541d772e111f675" exitCode=0
Sep 09 12:27:42.941603 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:42.941592    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6" event={"ID":"598d3ae5-2bec-49bb-8ed1-4a1a20454acb","Type":"ContainerDied","Data":"b48e2fb0d38e959df08294085ac5bc9dc46f4ce520232d5f4541d772e111f675"}
Sep 09 12:27:43.948686 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:43.948611    2582 generic.go:356] "Generic (PLEG): container finished" podID="598d3ae5-2bec-49bb-8ed1-4a1a20454acb" containerID="bb3945a5a744479136daf531db4dc7a7fce8428d00413f97cdcf67ca6403de51" exitCode=0
Sep 09 12:27:43.948686 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:43.948647    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6" event={"ID":"598d3ae5-2bec-49bb-8ed1-4a1a20454acb","Type":"ContainerDied","Data":"bb3945a5a744479136daf531db4dc7a7fce8428d00413f97cdcf67ca6403de51"}
Sep 09 12:27:43.950825 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:43.950801    2582 generic.go:356] "Generic (PLEG): container finished" podID="f35af65c-b570-4aa8-80a2-d11558f3c0e1" containerID="0a0598a676470aad4173fd481f9e3f030429b55db1c655c4bdbcebe3ede9ab7d" exitCode=0
Sep 09 12:27:43.950959 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:43.950880    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5" event={"ID":"f35af65c-b570-4aa8-80a2-d11558f3c0e1","Type":"ContainerDied","Data":"0a0598a676470aad4173fd481f9e3f030429b55db1c655c4bdbcebe3ede9ab7d"}
Sep 09 12:27:43.952511 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:43.952492    2582 generic.go:356] "Generic (PLEG): container finished" podID="4142cdaf-6fcf-4c64-bd49-e0816576268d" containerID="37f0708c914a1af884d8e9bdbf45fef4bcc66bef237d4afd02b5bbe08c92d6cb" exitCode=0
Sep 09 12:27:43.952574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:43.952528    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7" event={"ID":"4142cdaf-6fcf-4c64-bd49-e0816576268d","Type":"ContainerDied","Data":"37f0708c914a1af884d8e9bdbf45fef4bcc66bef237d4afd02b5bbe08c92d6cb"}
Sep 09 12:27:44.959963 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:44.959930    2582 generic.go:356] "Generic (PLEG): container finished" podID="4142cdaf-6fcf-4c64-bd49-e0816576268d" containerID="a395b99985cd04797f6db7006a124537c463bd4de438656957a957ba62d43f05" exitCode=0
Sep 09 12:27:44.960422 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:44.960016    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7" event={"ID":"4142cdaf-6fcf-4c64-bd49-e0816576268d","Type":"ContainerDied","Data":"a395b99985cd04797f6db7006a124537c463bd4de438656957a957ba62d43f05"}
Sep 09 12:27:44.961965 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:44.961941    2582 generic.go:356] "Generic (PLEG): container finished" podID="d76b69b1-1ab2-4bad-acc3-eaf81d760cbb" containerID="e30dbf272d1662b94e68c5e64eee415a60bdcd0ffe61fedf41704215231182c1" exitCode=0
Sep 09 12:27:44.962083 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:44.961974    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj" event={"ID":"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb","Type":"ContainerDied","Data":"e30dbf272d1662b94e68c5e64eee415a60bdcd0ffe61fedf41704215231182c1"}
Sep 09 12:27:45.099180 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.099156    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"
Sep 09 12:27:45.160388 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.160368    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"
Sep 09 12:27:45.199453 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.199429    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f35af65c-b570-4aa8-80a2-d11558f3c0e1-bundle\") pod \"f35af65c-b570-4aa8-80a2-d11558f3c0e1\" (UID: \"f35af65c-b570-4aa8-80a2-d11558f3c0e1\") "
Sep 09 12:27:45.199555 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.199478    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f35af65c-b570-4aa8-80a2-d11558f3c0e1-util\") pod \"f35af65c-b570-4aa8-80a2-d11558f3c0e1\" (UID: \"f35af65c-b570-4aa8-80a2-d11558f3c0e1\") "
Sep 09 12:27:45.199555 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.199525    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knkgt\" (UniqueName: \"kubernetes.io/projected/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-kube-api-access-knkgt\") pod \"598d3ae5-2bec-49bb-8ed1-4a1a20454acb\" (UID: \"598d3ae5-2bec-49bb-8ed1-4a1a20454acb\") "
Sep 09 12:27:45.199555 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.199540    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-util\") pod \"598d3ae5-2bec-49bb-8ed1-4a1a20454acb\" (UID: \"598d3ae5-2bec-49bb-8ed1-4a1a20454acb\") "
Sep 09 12:27:45.199703 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.199567    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-bundle\") pod \"598d3ae5-2bec-49bb-8ed1-4a1a20454acb\" (UID: \"598d3ae5-2bec-49bb-8ed1-4a1a20454acb\") "
Sep 09 12:27:45.199703 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.199599    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7mps\" (UniqueName: \"kubernetes.io/projected/f35af65c-b570-4aa8-80a2-d11558f3c0e1-kube-api-access-q7mps\") pod \"f35af65c-b570-4aa8-80a2-d11558f3c0e1\" (UID: \"f35af65c-b570-4aa8-80a2-d11558f3c0e1\") "
Sep 09 12:27:45.200135 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.200089    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f35af65c-b570-4aa8-80a2-d11558f3c0e1-bundle" (OuterVolumeSpecName: "bundle") pod "f35af65c-b570-4aa8-80a2-d11558f3c0e1" (UID: "f35af65c-b570-4aa8-80a2-d11558f3c0e1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:27:45.200399 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.200179    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-bundle" (OuterVolumeSpecName: "bundle") pod "598d3ae5-2bec-49bb-8ed1-4a1a20454acb" (UID: "598d3ae5-2bec-49bb-8ed1-4a1a20454acb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:27:45.201782 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.201747    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-kube-api-access-knkgt" (OuterVolumeSpecName: "kube-api-access-knkgt") pod "598d3ae5-2bec-49bb-8ed1-4a1a20454acb" (UID: "598d3ae5-2bec-49bb-8ed1-4a1a20454acb"). InnerVolumeSpecName "kube-api-access-knkgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:27:45.202249 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.202221    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f35af65c-b570-4aa8-80a2-d11558f3c0e1-kube-api-access-q7mps" (OuterVolumeSpecName: "kube-api-access-q7mps") pod "f35af65c-b570-4aa8-80a2-d11558f3c0e1" (UID: "f35af65c-b570-4aa8-80a2-d11558f3c0e1"). InnerVolumeSpecName "kube-api-access-q7mps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:27:45.205826 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.205805    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f35af65c-b570-4aa8-80a2-d11558f3c0e1-util" (OuterVolumeSpecName: "util") pod "f35af65c-b570-4aa8-80a2-d11558f3c0e1" (UID: "f35af65c-b570-4aa8-80a2-d11558f3c0e1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:27:45.206206 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.206184    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-util" (OuterVolumeSpecName: "util") pod "598d3ae5-2bec-49bb-8ed1-4a1a20454acb" (UID: "598d3ae5-2bec-49bb-8ed1-4a1a20454acb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:27:45.300368 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.300341    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f35af65c-b570-4aa8-80a2-d11558f3c0e1-util\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:27:45.300368 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.300363    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knkgt\" (UniqueName: \"kubernetes.io/projected/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-kube-api-access-knkgt\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:27:45.300368 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.300372    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-util\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:27:45.300529 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.300380    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/598d3ae5-2bec-49bb-8ed1-4a1a20454acb-bundle\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:27:45.300529 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.300388    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7mps\" (UniqueName: \"kubernetes.io/projected/f35af65c-b570-4aa8-80a2-d11558f3c0e1-kube-api-access-q7mps\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:27:45.300529 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.300395    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f35af65c-b570-4aa8-80a2-d11558f3c0e1-bundle\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:27:45.968866 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.968831    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5"
Sep 09 12:27:45.969262 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.968877    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6jxs5" event={"ID":"f35af65c-b570-4aa8-80a2-d11558f3c0e1","Type":"ContainerDied","Data":"e8cd55b5f090a3c44c30507ff8e6c8fd0ed66eb89806428868ecaface47846bd"}
Sep 09 12:27:45.969262 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.968930    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8cd55b5f090a3c44c30507ff8e6c8fd0ed66eb89806428868ecaface47846bd"
Sep 09 12:27:45.971359 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.971330    2582 generic.go:356] "Generic (PLEG): container finished" podID="d76b69b1-1ab2-4bad-acc3-eaf81d760cbb" containerID="aad610939ccdc1ab55836493401ce5db86eda8cb62ffd6d27836d7f8a82a8129" exitCode=0
Sep 09 12:27:45.971520 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.971408    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj" event={"ID":"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb","Type":"ContainerDied","Data":"aad610939ccdc1ab55836493401ce5db86eda8cb62ffd6d27836d7f8a82a8129"}
Sep 09 12:27:45.973450 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.973433    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6"
Sep 09 12:27:45.973555 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.973467    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7m6nc6" event={"ID":"598d3ae5-2bec-49bb-8ed1-4a1a20454acb","Type":"ContainerDied","Data":"9f60e884f302460b282d1a95de2f354b41caf69ee64512475ce7057ee1a5306c"}
Sep 09 12:27:45.973555 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:45.973499    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f60e884f302460b282d1a95de2f354b41caf69ee64512475ce7057ee1a5306c"
Sep 09 12:27:46.103734 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:46.103711    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"
Sep 09 12:27:46.208035 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:46.207974    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4142cdaf-6fcf-4c64-bd49-e0816576268d-util\") pod \"4142cdaf-6fcf-4c64-bd49-e0816576268d\" (UID: \"4142cdaf-6fcf-4c64-bd49-e0816576268d\") "
Sep 09 12:27:46.208194 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:46.208074    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7qj8\" (UniqueName: \"kubernetes.io/projected/4142cdaf-6fcf-4c64-bd49-e0816576268d-kube-api-access-j7qj8\") pod \"4142cdaf-6fcf-4c64-bd49-e0816576268d\" (UID: \"4142cdaf-6fcf-4c64-bd49-e0816576268d\") "
Sep 09 12:27:46.208194 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:46.208097    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4142cdaf-6fcf-4c64-bd49-e0816576268d-bundle\") pod \"4142cdaf-6fcf-4c64-bd49-e0816576268d\" (UID: \"4142cdaf-6fcf-4c64-bd49-e0816576268d\") "
Sep 09 12:27:46.208931 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:46.208900    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4142cdaf-6fcf-4c64-bd49-e0816576268d-bundle" (OuterVolumeSpecName: "bundle") pod "4142cdaf-6fcf-4c64-bd49-e0816576268d" (UID: "4142cdaf-6fcf-4c64-bd49-e0816576268d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:27:46.210370 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:46.210341    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4142cdaf-6fcf-4c64-bd49-e0816576268d-kube-api-access-j7qj8" (OuterVolumeSpecName: "kube-api-access-j7qj8") pod "4142cdaf-6fcf-4c64-bd49-e0816576268d" (UID: "4142cdaf-6fcf-4c64-bd49-e0816576268d"). InnerVolumeSpecName "kube-api-access-j7qj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:27:46.213759 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:46.213733    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4142cdaf-6fcf-4c64-bd49-e0816576268d-util" (OuterVolumeSpecName: "util") pod "4142cdaf-6fcf-4c64-bd49-e0816576268d" (UID: "4142cdaf-6fcf-4c64-bd49-e0816576268d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:27:46.309181 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:46.309154    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7qj8\" (UniqueName: \"kubernetes.io/projected/4142cdaf-6fcf-4c64-bd49-e0816576268d-kube-api-access-j7qj8\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:27:46.309181 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:46.309179    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4142cdaf-6fcf-4c64-bd49-e0816576268d-bundle\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:27:46.309341 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:46.309189    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4142cdaf-6fcf-4c64-bd49-e0816576268d-util\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:27:46.980261 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:46.980234    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7"
Sep 09 12:27:46.980261 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:46.980242    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ar6dp7" event={"ID":"4142cdaf-6fcf-4c64-bd49-e0816576268d","Type":"ContainerDied","Data":"5c2e214a14fd8402253a379fcdb98b3f7c5e8176ee0327866f99f0387af01a86"}
Sep 09 12:27:46.980751 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:46.980293    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c2e214a14fd8402253a379fcdb98b3f7c5e8176ee0327866f99f0387af01a86"
Sep 09 12:27:47.106650 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:47.106627    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"
Sep 09 12:27:47.217259 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:47.217232    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kctc2\" (UniqueName: \"kubernetes.io/projected/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-kube-api-access-kctc2\") pod \"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb\" (UID: \"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb\") "
Sep 09 12:27:47.217398 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:47.217278    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-util\") pod \"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb\" (UID: \"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb\") "
Sep 09 12:27:47.217473 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:47.217404    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-bundle\") pod \"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb\" (UID: \"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb\") "
Sep 09 12:27:47.217753 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:47.217732    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-bundle" (OuterVolumeSpecName: "bundle") pod "d76b69b1-1ab2-4bad-acc3-eaf81d760cbb" (UID: "d76b69b1-1ab2-4bad-acc3-eaf81d760cbb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:27:47.219352 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:47.219334    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-kube-api-access-kctc2" (OuterVolumeSpecName: "kube-api-access-kctc2") pod "d76b69b1-1ab2-4bad-acc3-eaf81d760cbb" (UID: "d76b69b1-1ab2-4bad-acc3-eaf81d760cbb"). InnerVolumeSpecName "kube-api-access-kctc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:27:47.306603 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:47.306557    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-util" (OuterVolumeSpecName: "util") pod "d76b69b1-1ab2-4bad-acc3-eaf81d760cbb" (UID: "d76b69b1-1ab2-4bad-acc3-eaf81d760cbb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:27:47.318840 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:47.318821    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-bundle\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:27:47.318920 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:47.318840    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kctc2\" (UniqueName: \"kubernetes.io/projected/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-kube-api-access-kctc2\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:27:47.318920 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:47.318867    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d76b69b1-1ab2-4bad-acc3-eaf81d760cbb-util\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:27:47.987783 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:47.987758    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj"
Sep 09 12:27:47.987783 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:47.987777    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vtscj" event={"ID":"d76b69b1-1ab2-4bad-acc3-eaf81d760cbb","Type":"ContainerDied","Data":"f4e65f42353e3cd9ec9aba2d761caed4e42f795e87f360fdc94392a0d6259ee8"}
Sep 09 12:27:47.988308 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:47.987821    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4e65f42353e3cd9ec9aba2d761caed4e42f795e87f360fdc94392a0d6259ee8"
Sep 09 12:27:55.148827 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.148790    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"]
Sep 09 12:27:55.149312 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149276    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f35af65c-b570-4aa8-80a2-d11558f3c0e1" containerName="extract"
Sep 09 12:27:55.149312 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149295    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f35af65c-b570-4aa8-80a2-d11558f3c0e1" containerName="extract"
Sep 09 12:27:55.149312 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149311    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d76b69b1-1ab2-4bad-acc3-eaf81d760cbb" containerName="util"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149320    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d76b69b1-1ab2-4bad-acc3-eaf81d760cbb" containerName="util"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149329    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="598d3ae5-2bec-49bb-8ed1-4a1a20454acb" containerName="util"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149337    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="598d3ae5-2bec-49bb-8ed1-4a1a20454acb" containerName="util"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149349    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="598d3ae5-2bec-49bb-8ed1-4a1a20454acb" containerName="extract"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149356    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="598d3ae5-2bec-49bb-8ed1-4a1a20454acb" containerName="extract"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149372    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4142cdaf-6fcf-4c64-bd49-e0816576268d" containerName="extract"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149380    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4142cdaf-6fcf-4c64-bd49-e0816576268d" containerName="extract"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149389    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d76b69b1-1ab2-4bad-acc3-eaf81d760cbb" containerName="extract"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149397    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d76b69b1-1ab2-4bad-acc3-eaf81d760cbb" containerName="extract"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149420    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4142cdaf-6fcf-4c64-bd49-e0816576268d" containerName="util"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149429    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4142cdaf-6fcf-4c64-bd49-e0816576268d" containerName="util"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149437    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="598d3ae5-2bec-49bb-8ed1-4a1a20454acb" containerName="pull"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149445    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="598d3ae5-2bec-49bb-8ed1-4a1a20454acb" containerName="pull"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149456    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4142cdaf-6fcf-4c64-bd49-e0816576268d" containerName="pull"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149463    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4142cdaf-6fcf-4c64-bd49-e0816576268d" containerName="pull"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149476    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f35af65c-b570-4aa8-80a2-d11558f3c0e1" containerName="pull"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149484    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f35af65c-b570-4aa8-80a2-d11558f3c0e1" containerName="pull"
Sep 09 12:27:55.149489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149493    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d76b69b1-1ab2-4bad-acc3-eaf81d760cbb" containerName="pull"
Sep 09 12:27:55.150286 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149501    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d76b69b1-1ab2-4bad-acc3-eaf81d760cbb" containerName="pull"
Sep 09 12:27:55.150286 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149510    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f35af65c-b570-4aa8-80a2-d11558f3c0e1" containerName="util"
Sep 09 12:27:55.150286 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149518    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f35af65c-b570-4aa8-80a2-d11558f3c0e1" containerName="util"
Sep 09 12:27:55.150286 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149600    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d76b69b1-1ab2-4bad-acc3-eaf81d760cbb" containerName="extract"
Sep 09 12:27:55.150286 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149612    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="598d3ae5-2bec-49bb-8ed1-4a1a20454acb" containerName="extract"
Sep 09 12:27:55.150286 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149624    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f35af65c-b570-4aa8-80a2-d11558f3c0e1" containerName="extract"
Sep 09 12:27:55.150286 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.149632    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4142cdaf-6fcf-4c64-bd49-e0816576268d" containerName="extract"
Sep 09 12:27:55.152483 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.152461    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:27:55.155160 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.155140    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 09 12:27:55.155289 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.155190    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2jbcm\""
Sep 09 12:27:55.155421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.155404    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 09 12:27:55.164503 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.164482    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"]
Sep 09 12:27:55.277424 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.277397    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/39bdcfbb-0787-4400-b11c-63d59390e839-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cgv6p\" (UID: \"39bdcfbb-0787-4400-b11c-63d59390e839\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:27:55.277560 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.277512    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b45sh\" (UniqueName: \"kubernetes.io/projected/39bdcfbb-0787-4400-b11c-63d59390e839-kube-api-access-b45sh\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cgv6p\" (UID: \"39bdcfbb-0787-4400-b11c-63d59390e839\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:27:55.378871 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.378817    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b45sh\" (UniqueName: \"kubernetes.io/projected/39bdcfbb-0787-4400-b11c-63d59390e839-kube-api-access-b45sh\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cgv6p\" (UID: \"39bdcfbb-0787-4400-b11c-63d59390e839\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:27:55.379032 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.378935    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/39bdcfbb-0787-4400-b11c-63d59390e839-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cgv6p\" (UID: \"39bdcfbb-0787-4400-b11c-63d59390e839\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:27:55.379359 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.379335    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/39bdcfbb-0787-4400-b11c-63d59390e839-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cgv6p\" (UID: \"39bdcfbb-0787-4400-b11c-63d59390e839\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:27:55.388303 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.388283    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b45sh\" (UniqueName: \"kubernetes.io/projected/39bdcfbb-0787-4400-b11c-63d59390e839-kube-api-access-b45sh\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cgv6p\" (UID: \"39bdcfbb-0787-4400-b11c-63d59390e839\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:27:55.463553 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.463475    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:27:55.744978 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:55.744953    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"]
Sep 09 12:27:55.747782 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:27:55.747756    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39bdcfbb_0787_4400_b11c_63d59390e839.slice/crio-d2974b14bb7aac8aa1a0db180e93a96849e5ae09d52085a72341b0efdeba6c5f WatchSource:0}: Error finding container d2974b14bb7aac8aa1a0db180e93a96849e5ae09d52085a72341b0efdeba6c5f: Status 404 returned error can't find the container with id d2974b14bb7aac8aa1a0db180e93a96849e5ae09d52085a72341b0efdeba6c5f
Sep 09 12:27:56.029366 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:56.029279    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p" event={"ID":"39bdcfbb-0787-4400-b11c-63d59390e839","Type":"ContainerStarted","Data":"d2974b14bb7aac8aa1a0db180e93a96849e5ae09d52085a72341b0efdeba6c5f"}
Sep 09 12:27:57.187072 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:57.185381    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-8lgh8"]
Sep 09 12:27:57.188141 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:57.188102    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-8lgh8"
Sep 09 12:27:57.192884 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:57.192833    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 09 12:27:57.193008 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:57.192947    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-m5f8s\""
Sep 09 12:27:57.206980 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:57.206955    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-8lgh8"]
Sep 09 12:27:57.293865 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:57.293823    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fwvr\" (UniqueName: \"kubernetes.io/projected/b173158e-31e1-4351-8cb6-4697d2260a17-kube-api-access-9fwvr\") pod \"dns-operator-controller-manager-7b84cd7c9c-8lgh8\" (UID: \"b173158e-31e1-4351-8cb6-4697d2260a17\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-8lgh8"
Sep 09 12:27:57.395398 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:57.395335    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fwvr\" (UniqueName: \"kubernetes.io/projected/b173158e-31e1-4351-8cb6-4697d2260a17-kube-api-access-9fwvr\") pod \"dns-operator-controller-manager-7b84cd7c9c-8lgh8\" (UID: \"b173158e-31e1-4351-8cb6-4697d2260a17\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-8lgh8"
Sep 09 12:27:57.405089 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:57.405060    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fwvr\" (UniqueName: \"kubernetes.io/projected/b173158e-31e1-4351-8cb6-4697d2260a17-kube-api-access-9fwvr\") pod \"dns-operator-controller-manager-7b84cd7c9c-8lgh8\" (UID: \"b173158e-31e1-4351-8cb6-4697d2260a17\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-8lgh8"
Sep 09 12:27:57.508070 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:57.507860    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-8lgh8"
Sep 09 12:27:58.432440 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:58.432312    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-8lgh8"]
Sep 09 12:27:59.055515 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:59.055471    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-8lgh8" event={"ID":"b173158e-31e1-4351-8cb6-4697d2260a17","Type":"ContainerStarted","Data":"4327101da12b6da570ba3bffbc1aa4f472419089218e1d2c292dcf97a38f6521"}
Sep 09 12:27:59.691286 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:59.691253    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"]
Sep 09 12:27:59.694342 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:59.694315    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:27:59.696718 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:59.696678    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-ndr52\""
Sep 09 12:27:59.707812 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:59.707788    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"]
Sep 09 12:27:59.817998 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:59.817966    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pwrd\" (UniqueName: \"kubernetes.io/projected/fbd737d4-0b8d-4188-8852-3a859a2b1aa2-kube-api-access-5pwrd\") pod \"limitador-operator-controller-manager-74495c69f8-s5gb9\" (UID: \"fbd737d4-0b8d-4188-8852-3a859a2b1aa2\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:27:59.918963 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:59.918893    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pwrd\" (UniqueName: \"kubernetes.io/projected/fbd737d4-0b8d-4188-8852-3a859a2b1aa2-kube-api-access-5pwrd\") pod \"limitador-operator-controller-manager-74495c69f8-s5gb9\" (UID: \"fbd737d4-0b8d-4188-8852-3a859a2b1aa2\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:27:59.933433 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:27:59.933406    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pwrd\" (UniqueName: \"kubernetes.io/projected/fbd737d4-0b8d-4188-8852-3a859a2b1aa2-kube-api-access-5pwrd\") pod \"limitador-operator-controller-manager-74495c69f8-s5gb9\" (UID: \"fbd737d4-0b8d-4188-8852-3a859a2b1aa2\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:28:00.008752 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:00.008727    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:28:00.063744 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:00.063710    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p" event={"ID":"39bdcfbb-0787-4400-b11c-63d59390e839","Type":"ContainerStarted","Data":"3a4c265d68fa14a6492dc36b7043e3bc08386d9ab8896ae42c791701bb316ae6"}
Sep 09 12:28:00.064046 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:00.064002    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:28:00.085543 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:00.085457    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p" podStartSLOduration=0.971630754 podStartE2EDuration="5.085349618s" totalImagesPullingTime="4.113718864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:28:00.083454747 +0000 UTC m=+499.780019903" watchObservedRunningTime="2026-09-09 12:28:00.085349618 +0000 UTC m=+499.781914772"
Sep 09 12:28:00.298350 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:00.298189    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"]
Sep 09 12:28:00.302004 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:28:00.301972    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd737d4_0b8d_4188_8852_3a859a2b1aa2.slice/crio-062868c84d1f45b331b11521cc6f7f793e874f0c304955166bb359f0a2ea8755 WatchSource:0}: Error finding container 062868c84d1f45b331b11521cc6f7f793e874f0c304955166bb359f0a2ea8755: Status 404 returned error can't find the container with id 062868c84d1f45b331b11521cc6f7f793e874f0c304955166bb359f0a2ea8755
Sep 09 12:28:01.072035 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:01.071929    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-8lgh8" event={"ID":"b173158e-31e1-4351-8cb6-4697d2260a17","Type":"ContainerStarted","Data":"40662fc9579fe23bd2d9fdd857f0bd77ca9ac87ba0fe3e6bf5739e0e6a81df05"}
Sep 09 12:28:01.072466 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:01.072058    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-8lgh8"
Sep 09 12:28:01.073529 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:01.073505    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9" event={"ID":"fbd737d4-0b8d-4188-8852-3a859a2b1aa2","Type":"ContainerStarted","Data":"062868c84d1f45b331b11521cc6f7f793e874f0c304955166bb359f0a2ea8755"}
Sep 09 12:28:01.091399 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:01.091339    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-8lgh8" podStartSLOduration=1.7533335989999999 podStartE2EDuration="4.091319157s" totalImagesPullingTime="2.337985558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:28:01.091104229 +0000 UTC m=+500.787669382" watchObservedRunningTime="2026-09-09 12:28:01.091319157 +0000 UTC m=+500.787884310"
Sep 09 12:28:02.292036 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:02.292002    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-g2s7l"]
Sep 09 12:28:02.295630 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:02.295608    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-g2s7l"
Sep 09 12:28:02.297807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:02.297782    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-gktdk\""
Sep 09 12:28:02.306681 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:02.306660    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-g2s7l"]
Sep 09 12:28:02.439989 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:02.439907    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml7m9\" (UniqueName: \"kubernetes.io/projected/b96aaf4b-8d2d-4a22-b0f5-6e902ed5b564-kube-api-access-ml7m9\") pod \"authorino-operator-76b478bd78-g2s7l\" (UID: \"b96aaf4b-8d2d-4a22-b0f5-6e902ed5b564\") " pod="kuadrant-system/authorino-operator-76b478bd78-g2s7l"
Sep 09 12:28:02.540870 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:02.540813    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml7m9\" (UniqueName: \"kubernetes.io/projected/b96aaf4b-8d2d-4a22-b0f5-6e902ed5b564-kube-api-access-ml7m9\") pod \"authorino-operator-76b478bd78-g2s7l\" (UID: \"b96aaf4b-8d2d-4a22-b0f5-6e902ed5b564\") " pod="kuadrant-system/authorino-operator-76b478bd78-g2s7l"
Sep 09 12:28:02.549534 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:02.549508    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml7m9\" (UniqueName: \"kubernetes.io/projected/b96aaf4b-8d2d-4a22-b0f5-6e902ed5b564-kube-api-access-ml7m9\") pod \"authorino-operator-76b478bd78-g2s7l\" (UID: \"b96aaf4b-8d2d-4a22-b0f5-6e902ed5b564\") " pod="kuadrant-system/authorino-operator-76b478bd78-g2s7l"
Sep 09 12:28:02.606148 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:02.606121    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-g2s7l"
Sep 09 12:28:02.882830 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:02.882799    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-g2s7l"]
Sep 09 12:28:02.884828 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:28:02.884798    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb96aaf4b_8d2d_4a22_b0f5_6e902ed5b564.slice/crio-b8b97032f5b35b4be79a80019dadaa53505f870fd01e2a47904cad4a14f3947d WatchSource:0}: Error finding container b8b97032f5b35b4be79a80019dadaa53505f870fd01e2a47904cad4a14f3947d: Status 404 returned error can't find the container with id b8b97032f5b35b4be79a80019dadaa53505f870fd01e2a47904cad4a14f3947d
Sep 09 12:28:03.087134 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:03.087098    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9" event={"ID":"fbd737d4-0b8d-4188-8852-3a859a2b1aa2","Type":"ContainerStarted","Data":"b0013e729f4daf094addb5b8666c9f0cd4a054c0434445443b4fbde25520a384"}
Sep 09 12:28:03.087314 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:03.087281    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:28:03.088494 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:03.088466    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-g2s7l" event={"ID":"b96aaf4b-8d2d-4a22-b0f5-6e902ed5b564","Type":"ContainerStarted","Data":"b8b97032f5b35b4be79a80019dadaa53505f870fd01e2a47904cad4a14f3947d"}
Sep 09 12:28:03.117180 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:03.117142    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9" podStartSLOduration=2.322513148 podStartE2EDuration="4.117128141s" totalImagesPullingTime="1.794614993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:28:03.115603657 +0000 UTC m=+502.812168809" watchObservedRunningTime="2026-09-09 12:28:03.117128141 +0000 UTC m=+502.813693293"
Sep 09 12:28:05.100898 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:05.100820    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-g2s7l" event={"ID":"b96aaf4b-8d2d-4a22-b0f5-6e902ed5b564","Type":"ContainerStarted","Data":"b3c6b019c23ddee0a8566a9eaec277aff9ecdfe7e745d15c9a631787eb767d26"}
Sep 09 12:28:05.101301 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:05.100979    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-g2s7l"
Sep 09 12:28:05.119468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:05.119423    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-g2s7l" podStartSLOduration=1.601999276 podStartE2EDuration="3.119406821s" totalImagesPullingTime="1.517407545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:28:05.118687265 +0000 UTC m=+504.815252436" watchObservedRunningTime="2026-09-09 12:28:05.119406821 +0000 UTC m=+504.815971974"
Sep 09 12:28:11.075987 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:11.075960    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:28:12.432131 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:12.432100    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"]
Sep 09 12:28:12.435944 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:12.435925    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:12.449966 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:12.449946    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"]
Sep 09 12:28:12.527441 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:12.527417    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/424c2817-8c53-489a-89d9-9f42c1a6cc4e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-wdl8n\" (UID: \"424c2817-8c53-489a-89d9-9f42c1a6cc4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:12.527558 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:12.527484    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f56wm\" (UniqueName: \"kubernetes.io/projected/424c2817-8c53-489a-89d9-9f42c1a6cc4e-kube-api-access-f56wm\") pod \"kuadrant-operator-controller-manager-5f685c496f-wdl8n\" (UID: \"424c2817-8c53-489a-89d9-9f42c1a6cc4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:12.628274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:12.628196    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/424c2817-8c53-489a-89d9-9f42c1a6cc4e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-wdl8n\" (UID: \"424c2817-8c53-489a-89d9-9f42c1a6cc4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:12.628386 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:12.628345    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f56wm\" (UniqueName: \"kubernetes.io/projected/424c2817-8c53-489a-89d9-9f42c1a6cc4e-kube-api-access-f56wm\") pod \"kuadrant-operator-controller-manager-5f685c496f-wdl8n\" (UID: \"424c2817-8c53-489a-89d9-9f42c1a6cc4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:12.628556 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:12.628536    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/424c2817-8c53-489a-89d9-9f42c1a6cc4e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-wdl8n\" (UID: \"424c2817-8c53-489a-89d9-9f42c1a6cc4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:12.642051 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:12.642027    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f56wm\" (UniqueName: \"kubernetes.io/projected/424c2817-8c53-489a-89d9-9f42c1a6cc4e-kube-api-access-f56wm\") pod \"kuadrant-operator-controller-manager-5f685c496f-wdl8n\" (UID: \"424c2817-8c53-489a-89d9-9f42c1a6cc4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:12.745954 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:12.745928    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:13.020828 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.020755    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"]
Sep 09 12:28:13.023915 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:28:13.023885    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod424c2817_8c53_489a_89d9_9f42c1a6cc4e.slice/crio-9b0f9149ac5867d4b326103282e2ab3915afaedfb1b04cde798ba4c883f4e746 WatchSource:0}: Error finding container 9b0f9149ac5867d4b326103282e2ab3915afaedfb1b04cde798ba4c883f4e746: Status 404 returned error can't find the container with id 9b0f9149ac5867d4b326103282e2ab3915afaedfb1b04cde798ba4c883f4e746
Sep 09 12:28:13.090693 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.090668    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-8lgh8"
Sep 09 12:28:13.146025 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.145974    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n" event={"ID":"424c2817-8c53-489a-89d9-9f42c1a6cc4e","Type":"ContainerStarted","Data":"b8d2b5928d9ea2682991d35f4ddb5e6abc52a1a6e8dc34322def8a218bf59e06"}
Sep 09 12:28:13.146173 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.146036    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n" event={"ID":"424c2817-8c53-489a-89d9-9f42c1a6cc4e","Type":"ContainerStarted","Data":"9b0f9149ac5867d4b326103282e2ab3915afaedfb1b04cde798ba4c883f4e746"}
Sep 09 12:28:13.146173 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.146091    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:13.163247 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.163200    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n" podStartSLOduration=1.163185792 podStartE2EDuration="1.163185792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:28:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:28:13.161924674 +0000 UTC m=+512.858489807" watchObservedRunningTime="2026-09-09 12:28:13.163185792 +0000 UTC m=+512.859750944"
Sep 09 12:28:13.481607 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.481522    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"]
Sep 09 12:28:13.491826 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.491798    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"]
Sep 09 12:28:13.507415 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.507379    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"]
Sep 09 12:28:13.507948 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.507932    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="424c2817-8c53-489a-89d9-9f42c1a6cc4e" containerName="manager"
Sep 09 12:28:13.508000 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.507953    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="424c2817-8c53-489a-89d9-9f42c1a6cc4e" containerName="manager"
Sep 09 12:28:13.508076 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.508065    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="424c2817-8c53-489a-89d9-9f42c1a6cc4e" containerName="manager"
Sep 09 12:28:13.511623 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.511603    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"]
Sep 09 12:28:13.511727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.511709    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"
Sep 09 12:28:13.511801 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.511780    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p" podUID="39bdcfbb-0787-4400-b11c-63d59390e839" containerName="manager" containerID="cri-o://3a4c265d68fa14a6492dc36b7043e3bc08386d9ab8896ae42c791701bb316ae6" gracePeriod=2
Sep 09 12:28:13.528022 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.527977    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"]
Sep 09 12:28:13.530673 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.530646    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"]
Sep 09 12:28:13.539141 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.539115    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"]
Sep 09 12:28:13.539636 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.539622    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39bdcfbb-0787-4400-b11c-63d59390e839" containerName="manager"
Sep 09 12:28:13.539685 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.539641    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="39bdcfbb-0787-4400-b11c-63d59390e839" containerName="manager"
Sep 09 12:28:13.539739 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.539728    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="39bdcfbb-0787-4400-b11c-63d59390e839" containerName="manager"
Sep 09 12:28:13.543139 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.543120    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"]
Sep 09 12:28:13.543235 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.543218    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"
Sep 09 12:28:13.543402 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.543369    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9" podUID="fbd737d4-0b8d-4188-8852-3a859a2b1aa2" containerName="manager" containerID="cri-o://b0013e729f4daf094addb5b8666c9f0cd4a054c0434445443b4fbde25520a384" gracePeriod=2
Sep 09 12:28:13.545285 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.545266    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:28:13.548945 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:13.548912    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-cgv6p\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="39bdcfbb-0787-4400-b11c-63d59390e839" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:28:13.554243 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.554219    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"]
Sep 09 12:28:13.557966 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.557942    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"]
Sep 09 12:28:13.568610 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.568592    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-gjpwp"]
Sep 09 12:28:13.569214 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.569192    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbd737d4-0b8d-4188-8852-3a859a2b1aa2" containerName="manager"
Sep 09 12:28:13.569214 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.569214    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbd737d4-0b8d-4188-8852-3a859a2b1aa2" containerName="manager"
Sep 09 12:28:13.569374 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.569292    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbd737d4-0b8d-4188-8852-3a859a2b1aa2" containerName="manager"
Sep 09 12:28:13.571296 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:13.571146    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-s5gb9\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="fbd737d4-0b8d-4188-8852-3a859a2b1aa2" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:28:13.572921 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.572903    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gjpwp"
Sep 09 12:28:13.575094 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:13.575068    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-s5gb9\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="fbd737d4-0b8d-4188-8852-3a859a2b1aa2" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:28:13.576682 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:13.576656    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-cgv6p\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="39bdcfbb-0787-4400-b11c-63d59390e839" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:28:13.581249 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.581234    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-gjpwp"]
Sep 09 12:28:13.636473 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.636445    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66445b0b-2779-4527-b53b-08dcedd60c95-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-nwpb2\" (UID: \"66445b0b-2779-4527-b53b-08dcedd60c95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"
Sep 09 12:28:13.636604 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.636477    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/30df6f0a-26be-4d97-b1e0-d344a42c257e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9zhg2\" (UID: \"30df6f0a-26be-4d97-b1e0-d344a42c257e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"
Sep 09 12:28:13.636604 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.636496    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhkhj\" (UniqueName: \"kubernetes.io/projected/30df6f0a-26be-4d97-b1e0-d344a42c257e-kube-api-access-mhkhj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9zhg2\" (UID: \"30df6f0a-26be-4d97-b1e0-d344a42c257e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"
Sep 09 12:28:13.636691 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.636642    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tpd4\" (UniqueName: \"kubernetes.io/projected/66445b0b-2779-4527-b53b-08dcedd60c95-kube-api-access-6tpd4\") pod \"kuadrant-operator-controller-manager-5f685c496f-nwpb2\" (UID: \"66445b0b-2779-4527-b53b-08dcedd60c95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"
Sep 09 12:28:13.737429 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.737361    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tpd4\" (UniqueName: \"kubernetes.io/projected/66445b0b-2779-4527-b53b-08dcedd60c95-kube-api-access-6tpd4\") pod \"kuadrant-operator-controller-manager-5f685c496f-nwpb2\" (UID: \"66445b0b-2779-4527-b53b-08dcedd60c95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"
Sep 09 12:28:13.737429 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.737427    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66445b0b-2779-4527-b53b-08dcedd60c95-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-nwpb2\" (UID: \"66445b0b-2779-4527-b53b-08dcedd60c95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"
Sep 09 12:28:13.737618 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.737443    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/30df6f0a-26be-4d97-b1e0-d344a42c257e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9zhg2\" (UID: \"30df6f0a-26be-4d97-b1e0-d344a42c257e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"
Sep 09 12:28:13.737618 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.737458    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhkhj\" (UniqueName: \"kubernetes.io/projected/30df6f0a-26be-4d97-b1e0-d344a42c257e-kube-api-access-mhkhj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9zhg2\" (UID: \"30df6f0a-26be-4d97-b1e0-d344a42c257e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"
Sep 09 12:28:13.737618 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.737482    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xjwv\" (UniqueName: \"kubernetes.io/projected/ac115547-fe26-4e18-92ef-1cb8093ac78a-kube-api-access-2xjwv\") pod \"limitador-operator-controller-manager-74495c69f8-gjpwp\" (UID: \"ac115547-fe26-4e18-92ef-1cb8093ac78a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gjpwp"
Sep 09 12:28:13.737933 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.737894    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/30df6f0a-26be-4d97-b1e0-d344a42c257e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9zhg2\" (UID: \"30df6f0a-26be-4d97-b1e0-d344a42c257e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"
Sep 09 12:28:13.738037 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.737914    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66445b0b-2779-4527-b53b-08dcedd60c95-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-nwpb2\" (UID: \"66445b0b-2779-4527-b53b-08dcedd60c95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"
Sep 09 12:28:13.746468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.746437    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tpd4\" (UniqueName: \"kubernetes.io/projected/66445b0b-2779-4527-b53b-08dcedd60c95-kube-api-access-6tpd4\") pod \"kuadrant-operator-controller-manager-5f685c496f-nwpb2\" (UID: \"66445b0b-2779-4527-b53b-08dcedd60c95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"
Sep 09 12:28:13.746991 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.746971    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhkhj\" (UniqueName: \"kubernetes.io/projected/30df6f0a-26be-4d97-b1e0-d344a42c257e-kube-api-access-mhkhj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9zhg2\" (UID: \"30df6f0a-26be-4d97-b1e0-d344a42c257e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"
Sep 09 12:28:13.784423 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.784403    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:28:13.786110 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:13.786087    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-cgv6p\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="39bdcfbb-0787-4400-b11c-63d59390e839" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:28:13.786368 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.786355    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:28:13.787490 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:13.787468    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-s5gb9\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="fbd737d4-0b8d-4188-8852-3a859a2b1aa2" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:28:13.788843 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:13.788824    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-cgv6p\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="39bdcfbb-0787-4400-b11c-63d59390e839" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:28:13.790158 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:13.790139    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-s5gb9\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="fbd737d4-0b8d-4188-8852-3a859a2b1aa2" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:28:13.826762 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.826743    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"
Sep 09 12:28:13.838666 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.838649    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xjwv\" (UniqueName: \"kubernetes.io/projected/ac115547-fe26-4e18-92ef-1cb8093ac78a-kube-api-access-2xjwv\") pod \"limitador-operator-controller-manager-74495c69f8-gjpwp\" (UID: \"ac115547-fe26-4e18-92ef-1cb8093ac78a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gjpwp"
Sep 09 12:28:13.851961 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.851933    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xjwv\" (UniqueName: \"kubernetes.io/projected/ac115547-fe26-4e18-92ef-1cb8093ac78a-kube-api-access-2xjwv\") pod \"limitador-operator-controller-manager-74495c69f8-gjpwp\" (UID: \"ac115547-fe26-4e18-92ef-1cb8093ac78a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gjpwp"
Sep 09 12:28:13.939173 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.939142    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/39bdcfbb-0787-4400-b11c-63d59390e839-extensions-socket-volume\") pod \"39bdcfbb-0787-4400-b11c-63d59390e839\" (UID: \"39bdcfbb-0787-4400-b11c-63d59390e839\") "
Sep 09 12:28:13.939332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.939208    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pwrd\" (UniqueName: \"kubernetes.io/projected/fbd737d4-0b8d-4188-8852-3a859a2b1aa2-kube-api-access-5pwrd\") pod \"fbd737d4-0b8d-4188-8852-3a859a2b1aa2\" (UID: \"fbd737d4-0b8d-4188-8852-3a859a2b1aa2\") "
Sep 09 12:28:13.939332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.939302    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b45sh\" (UniqueName: \"kubernetes.io/projected/39bdcfbb-0787-4400-b11c-63d59390e839-kube-api-access-b45sh\") pod \"39bdcfbb-0787-4400-b11c-63d59390e839\" (UID: \"39bdcfbb-0787-4400-b11c-63d59390e839\") "
Sep 09 12:28:13.939655 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.939628    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39bdcfbb-0787-4400-b11c-63d59390e839-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "39bdcfbb-0787-4400-b11c-63d59390e839" (UID: "39bdcfbb-0787-4400-b11c-63d59390e839"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:28:13.941498 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.941475    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbd737d4-0b8d-4188-8852-3a859a2b1aa2-kube-api-access-5pwrd" (OuterVolumeSpecName: "kube-api-access-5pwrd") pod "fbd737d4-0b8d-4188-8852-3a859a2b1aa2" (UID: "fbd737d4-0b8d-4188-8852-3a859a2b1aa2"). InnerVolumeSpecName "kube-api-access-5pwrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:28:13.941559 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.941500    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39bdcfbb-0787-4400-b11c-63d59390e839-kube-api-access-b45sh" (OuterVolumeSpecName: "kube-api-access-b45sh") pod "39bdcfbb-0787-4400-b11c-63d59390e839" (UID: "39bdcfbb-0787-4400-b11c-63d59390e839"). InnerVolumeSpecName "kube-api-access-b45sh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:28:13.956731 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.956706    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"
Sep 09 12:28:13.963826 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:13.963811    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gjpwp"
Sep 09 12:28:14.040453 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.040426    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b45sh\" (UniqueName: \"kubernetes.io/projected/39bdcfbb-0787-4400-b11c-63d59390e839-kube-api-access-b45sh\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:28:14.040453 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.040451    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/39bdcfbb-0787-4400-b11c-63d59390e839-extensions-socket-volume\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:28:14.040635 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.040461    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pwrd\" (UniqueName: \"kubernetes.io/projected/fbd737d4-0b8d-4188-8852-3a859a2b1aa2-kube-api-access-5pwrd\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:28:14.114280 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.114260    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"]
Sep 09 12:28:14.116295 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:28:14.116270    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66445b0b_2779_4527_b53b_08dcedd60c95.slice/crio-ebc906c500d31e0c1c39def99747c5041d7fd47aca79ff31169433a172d20600 WatchSource:0}: Error finding container ebc906c500d31e0c1c39def99747c5041d7fd47aca79ff31169433a172d20600: Status 404 returned error can't find the container with id ebc906c500d31e0c1c39def99747c5041d7fd47aca79ff31169433a172d20600
Sep 09 12:28:14.152402 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.152378    2582 generic.go:356] "Generic (PLEG): container finished" podID="fbd737d4-0b8d-4188-8852-3a859a2b1aa2" containerID="b0013e729f4daf094addb5b8666c9f0cd4a054c0434445443b4fbde25520a384" exitCode=0
Sep 09 12:28:14.152492 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.152413    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:28:14.152492 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.152489    2582 scope.go:117] "RemoveContainer" containerID="b0013e729f4daf094addb5b8666c9f0cd4a054c0434445443b4fbde25520a384"
Sep 09 12:28:14.154526 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.154399    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-cgv6p\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="39bdcfbb-0787-4400-b11c-63d59390e839" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:28:14.154723 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.154212    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2" event={"ID":"66445b0b-2779-4527-b53b-08dcedd60c95","Type":"ContainerStarted","Data":"ebc906c500d31e0c1c39def99747c5041d7fd47aca79ff31169433a172d20600"}
Sep 09 12:28:14.156116 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.156078    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-s5gb9\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="fbd737d4-0b8d-4188-8852-3a859a2b1aa2" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:28:14.156290 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.156268    2582 generic.go:356] "Generic (PLEG): container finished" podID="39bdcfbb-0787-4400-b11c-63d59390e839" containerID="3a4c265d68fa14a6492dc36b7043e3bc08386d9ab8896ae42c791701bb316ae6" exitCode=0
Sep 09 12:28:14.156358 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.156313    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:28:14.156699 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.156678    2582 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n" container="manager"
Sep 09 12:28:14.158242 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.158216    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-s5gb9\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="fbd737d4-0b8d-4188-8852-3a859a2b1aa2" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:28:14.160077 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.160031    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-cgv6p\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="39bdcfbb-0787-4400-b11c-63d59390e839" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:28:14.161575 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.161546    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-cgv6p\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="39bdcfbb-0787-4400-b11c-63d59390e839" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:28:14.162893 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.162869    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-s5gb9\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="fbd737d4-0b8d-4188-8852-3a859a2b1aa2" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:28:14.164292 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.164263    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-wdl8n\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="424c2817-8c53-489a-89d9-9f42c1a6cc4e" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:14.170710 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.170678    2582 scope.go:117] "RemoveContainer" containerID="b0013e729f4daf094addb5b8666c9f0cd4a054c0434445443b4fbde25520a384"
Sep 09 12:28:14.171034 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.171015    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0013e729f4daf094addb5b8666c9f0cd4a054c0434445443b4fbde25520a384\": container with ID starting with b0013e729f4daf094addb5b8666c9f0cd4a054c0434445443b4fbde25520a384 not found: ID does not exist" containerID="b0013e729f4daf094addb5b8666c9f0cd4a054c0434445443b4fbde25520a384"
Sep 09 12:28:14.171113 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.171045    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0013e729f4daf094addb5b8666c9f0cd4a054c0434445443b4fbde25520a384"} err="failed to get container status \"b0013e729f4daf094addb5b8666c9f0cd4a054c0434445443b4fbde25520a384\": rpc error: code = NotFound desc = could not find container \"b0013e729f4daf094addb5b8666c9f0cd4a054c0434445443b4fbde25520a384\": container with ID starting with b0013e729f4daf094addb5b8666c9f0cd4a054c0434445443b4fbde25520a384 not found: ID does not exist"
Sep 09 12:28:14.171113 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.171067    2582 scope.go:117] "RemoveContainer" containerID="3a4c265d68fa14a6492dc36b7043e3bc08386d9ab8896ae42c791701bb316ae6"
Sep 09 12:28:14.178200 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.178173    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-cgv6p\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="39bdcfbb-0787-4400-b11c-63d59390e839" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:28:14.179644 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.179607    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-s5gb9\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="fbd737d4-0b8d-4188-8852-3a859a2b1aa2" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:28:14.181068 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.181002    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-wdl8n\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="424c2817-8c53-489a-89d9-9f42c1a6cc4e" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:14.181136 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.181119    2582 scope.go:117] "RemoveContainer" containerID="3a4c265d68fa14a6492dc36b7043e3bc08386d9ab8896ae42c791701bb316ae6"
Sep 09 12:28:14.181441 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.181424    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a4c265d68fa14a6492dc36b7043e3bc08386d9ab8896ae42c791701bb316ae6\": container with ID starting with 3a4c265d68fa14a6492dc36b7043e3bc08386d9ab8896ae42c791701bb316ae6 not found: ID does not exist" containerID="3a4c265d68fa14a6492dc36b7043e3bc08386d9ab8896ae42c791701bb316ae6"
Sep 09 12:28:14.181498 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.181451    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a4c265d68fa14a6492dc36b7043e3bc08386d9ab8896ae42c791701bb316ae6"} err="failed to get container status \"3a4c265d68fa14a6492dc36b7043e3bc08386d9ab8896ae42c791701bb316ae6\": rpc error: code = NotFound desc = could not find container \"3a4c265d68fa14a6492dc36b7043e3bc08386d9ab8896ae42c791701bb316ae6\": container with ID starting with 3a4c265d68fa14a6492dc36b7043e3bc08386d9ab8896ae42c791701bb316ae6 not found: ID does not exist"
Sep 09 12:28:14.182638 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.182615    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-s5gb9\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="fbd737d4-0b8d-4188-8852-3a859a2b1aa2" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-s5gb9"
Sep 09 12:28:14.184003 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.183977    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-wdl8n\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="424c2817-8c53-489a-89d9-9f42c1a6cc4e" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:14.185292 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:14.185265    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-cgv6p\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="39bdcfbb-0787-4400-b11c-63d59390e839" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgv6p"
Sep 09 12:28:14.242324 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.242295    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"]
Sep 09 12:28:14.243217 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:28:14.243100    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30df6f0a_26be_4d97_b1e0_d344a42c257e.slice/crio-25fb5996695ebf4d7a0bdbf2c680996e3c5d2628a6c6580cf948f9402a9ff51e WatchSource:0}: Error finding container 25fb5996695ebf4d7a0bdbf2c680996e3c5d2628a6c6580cf948f9402a9ff51e: Status 404 returned error can't find the container with id 25fb5996695ebf4d7a0bdbf2c680996e3c5d2628a6c6580cf948f9402a9ff51e
Sep 09 12:28:14.244524 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.244502    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-gjpwp"]
Sep 09 12:28:14.245679 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:28:14.245657    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac115547_fe26_4e18_92ef_1cb8093ac78a.slice/crio-abf0c6944946a4df2c855442608b3a29830a716de15f539181d5faa1e06a5b10 WatchSource:0}: Error finding container abf0c6944946a4df2c855442608b3a29830a716de15f539181d5faa1e06a5b10: Status 404 returned error can't find the container with id abf0c6944946a4df2c855442608b3a29830a716de15f539181d5faa1e06a5b10
Sep 09 12:28:14.826239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.826199    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39bdcfbb-0787-4400-b11c-63d59390e839" path="/var/lib/kubelet/pods/39bdcfbb-0787-4400-b11c-63d59390e839/volumes"
Sep 09 12:28:14.826677 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:14.826651    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbd737d4-0b8d-4188-8852-3a859a2b1aa2" path="/var/lib/kubelet/pods/fbd737d4-0b8d-4188-8852-3a859a2b1aa2/volumes"
Sep 09 12:28:15.172515 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.172347    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2" event={"ID":"30df6f0a-26be-4d97-b1e0-d344a42c257e","Type":"ContainerStarted","Data":"a260015d29466141c5e15c034d6ffaacc740bc78e410f7b73b175001335774b2"}
Sep 09 12:28:15.172515 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.172442    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2" event={"ID":"30df6f0a-26be-4d97-b1e0-d344a42c257e","Type":"ContainerStarted","Data":"25fb5996695ebf4d7a0bdbf2c680996e3c5d2628a6c6580cf948f9402a9ff51e"}
Sep 09 12:28:15.172515 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.172476    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"
Sep 09 12:28:15.174567 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.174527    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gjpwp" event={"ID":"ac115547-fe26-4e18-92ef-1cb8093ac78a","Type":"ContainerStarted","Data":"be19eeb69b286755976be3a37a9ea48bfe34924c9b4bf8799ea9086924dd59a2"}
Sep 09 12:28:15.174696 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.174587    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gjpwp" event={"ID":"ac115547-fe26-4e18-92ef-1cb8093ac78a","Type":"ContainerStarted","Data":"abf0c6944946a4df2c855442608b3a29830a716de15f539181d5faa1e06a5b10"}
Sep 09 12:28:15.174696 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.174614    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gjpwp"
Sep 09 12:28:15.178076 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.178049    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2" event={"ID":"66445b0b-2779-4527-b53b-08dcedd60c95","Type":"ContainerStarted","Data":"ab9567c3b34215ac21abae4ce0b12296de2a6e3ef5c07ceb3f086439a1b267fa"}
Sep 09 12:28:15.178171 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.178153    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"
Sep 09 12:28:15.180064 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.180039    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n" podUID="424c2817-8c53-489a-89d9-9f42c1a6cc4e" containerName="manager" containerID="cri-o://b8d2b5928d9ea2682991d35f4ddb5e6abc52a1a6e8dc34322def8a218bf59e06" gracePeriod=2
Sep 09 12:28:15.198375 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.198314    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2" podStartSLOduration=2.198297223 podStartE2EDuration="2.198297223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:28:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:28:15.197706033 +0000 UTC m=+514.894271184" watchObservedRunningTime="2026-09-09 12:28:15.198297223 +0000 UTC m=+514.894862376"
Sep 09 12:28:15.224204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.224155    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gjpwp" podStartSLOduration=2.224142074 podStartE2EDuration="2.224142074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:28:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:28:15.223353372 +0000 UTC m=+514.919918552" watchObservedRunningTime="2026-09-09 12:28:15.224142074 +0000 UTC m=+514.920707226"
Sep 09 12:28:15.247553 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.247512    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2" podStartSLOduration=2.247500104 podStartE2EDuration="2.247500104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:28:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:28:15.245482229 +0000 UTC m=+514.942047384" watchObservedRunningTime="2026-09-09 12:28:15.247500104 +0000 UTC m=+514.944065255"
Sep 09 12:28:15.423602 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.423549    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:15.554316 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.554288    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f56wm\" (UniqueName: \"kubernetes.io/projected/424c2817-8c53-489a-89d9-9f42c1a6cc4e-kube-api-access-f56wm\") pod \"424c2817-8c53-489a-89d9-9f42c1a6cc4e\" (UID: \"424c2817-8c53-489a-89d9-9f42c1a6cc4e\") "
Sep 09 12:28:15.554491 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.554354    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/424c2817-8c53-489a-89d9-9f42c1a6cc4e-extensions-socket-volume\") pod \"424c2817-8c53-489a-89d9-9f42c1a6cc4e\" (UID: \"424c2817-8c53-489a-89d9-9f42c1a6cc4e\") "
Sep 09 12:28:15.554727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.554694    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/424c2817-8c53-489a-89d9-9f42c1a6cc4e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "424c2817-8c53-489a-89d9-9f42c1a6cc4e" (UID: "424c2817-8c53-489a-89d9-9f42c1a6cc4e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:28:15.556500 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.556479    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/424c2817-8c53-489a-89d9-9f42c1a6cc4e-kube-api-access-f56wm" (OuterVolumeSpecName: "kube-api-access-f56wm") pod "424c2817-8c53-489a-89d9-9f42c1a6cc4e" (UID: "424c2817-8c53-489a-89d9-9f42c1a6cc4e"). InnerVolumeSpecName "kube-api-access-f56wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:28:15.655443 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.655396    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f56wm\" (UniqueName: \"kubernetes.io/projected/424c2817-8c53-489a-89d9-9f42c1a6cc4e-kube-api-access-f56wm\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:28:15.655443 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:15.655437    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/424c2817-8c53-489a-89d9-9f42c1a6cc4e-extensions-socket-volume\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:28:16.108160 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:16.108124    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-g2s7l"
Sep 09 12:28:16.188542 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:16.188511    2582 generic.go:356] "Generic (PLEG): container finished" podID="424c2817-8c53-489a-89d9-9f42c1a6cc4e" containerID="b8d2b5928d9ea2682991d35f4ddb5e6abc52a1a6e8dc34322def8a218bf59e06" exitCode=0
Sep 09 12:28:16.188698 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:16.188556    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:16.188698 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:16.188610    2582 scope.go:117] "RemoveContainer" containerID="b8d2b5928d9ea2682991d35f4ddb5e6abc52a1a6e8dc34322def8a218bf59e06"
Sep 09 12:28:16.191170 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:16.191140    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-wdl8n\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="424c2817-8c53-489a-89d9-9f42c1a6cc4e" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:16.198455 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:16.198438    2582 scope.go:117] "RemoveContainer" containerID="b8d2b5928d9ea2682991d35f4ddb5e6abc52a1a6e8dc34322def8a218bf59e06"
Sep 09 12:28:16.198771 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:16.198752    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8d2b5928d9ea2682991d35f4ddb5e6abc52a1a6e8dc34322def8a218bf59e06\": container with ID starting with b8d2b5928d9ea2682991d35f4ddb5e6abc52a1a6e8dc34322def8a218bf59e06 not found: ID does not exist" containerID="b8d2b5928d9ea2682991d35f4ddb5e6abc52a1a6e8dc34322def8a218bf59e06"
Sep 09 12:28:16.198825 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:16.198779    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8d2b5928d9ea2682991d35f4ddb5e6abc52a1a6e8dc34322def8a218bf59e06"} err="failed to get container status \"b8d2b5928d9ea2682991d35f4ddb5e6abc52a1a6e8dc34322def8a218bf59e06\": rpc error: code = NotFound desc = could not find container \"b8d2b5928d9ea2682991d35f4ddb5e6abc52a1a6e8dc34322def8a218bf59e06\": container with ID starting with b8d2b5928d9ea2682991d35f4ddb5e6abc52a1a6e8dc34322def8a218bf59e06 not found: ID does not exist"
Sep 09 12:28:16.199760 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:16.199737    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-wdl8n\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object" podUID="424c2817-8c53-489a-89d9-9f42c1a6cc4e" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wdl8n"
Sep 09 12:28:16.825117 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:16.825089    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="424c2817-8c53-489a-89d9-9f42c1a6cc4e" path="/var/lib/kubelet/pods/424c2817-8c53-489a-89d9-9f42c1a6cc4e/volumes"
Sep 09 12:28:26.191118 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.191087    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"
Sep 09 12:28:26.191555 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.191325    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"
Sep 09 12:28:26.191555 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.191403    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gjpwp"
Sep 09 12:28:26.264998 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.264971    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"]
Sep 09 12:28:26.265165 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.265143    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2" podUID="30df6f0a-26be-4d97-b1e0-d344a42c257e" containerName="manager" containerID="cri-o://a260015d29466141c5e15c034d6ffaacc740bc78e410f7b73b175001335774b2" gracePeriod=10
Sep 09 12:28:26.517995 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.517967    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"
Sep 09 12:28:26.544577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.544554    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/30df6f0a-26be-4d97-b1e0-d344a42c257e-extensions-socket-volume\") pod \"30df6f0a-26be-4d97-b1e0-d344a42c257e\" (UID: \"30df6f0a-26be-4d97-b1e0-d344a42c257e\") "
Sep 09 12:28:26.544672 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.544590    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhkhj\" (UniqueName: \"kubernetes.io/projected/30df6f0a-26be-4d97-b1e0-d344a42c257e-kube-api-access-mhkhj\") pod \"30df6f0a-26be-4d97-b1e0-d344a42c257e\" (UID: \"30df6f0a-26be-4d97-b1e0-d344a42c257e\") "
Sep 09 12:28:26.544900 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.544877    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30df6f0a-26be-4d97-b1e0-d344a42c257e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "30df6f0a-26be-4d97-b1e0-d344a42c257e" (UID: "30df6f0a-26be-4d97-b1e0-d344a42c257e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:28:26.546704 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.546684    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30df6f0a-26be-4d97-b1e0-d344a42c257e-kube-api-access-mhkhj" (OuterVolumeSpecName: "kube-api-access-mhkhj") pod "30df6f0a-26be-4d97-b1e0-d344a42c257e" (UID: "30df6f0a-26be-4d97-b1e0-d344a42c257e"). InnerVolumeSpecName "kube-api-access-mhkhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:28:26.645752 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.645723    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/30df6f0a-26be-4d97-b1e0-d344a42c257e-extensions-socket-volume\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:28:26.645752 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.645748    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhkhj\" (UniqueName: \"kubernetes.io/projected/30df6f0a-26be-4d97-b1e0-d344a42c257e-kube-api-access-mhkhj\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:28:26.651696 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.651672    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"]
Sep 09 12:28:26.652077 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.652064    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30df6f0a-26be-4d97-b1e0-d344a42c257e" containerName="manager"
Sep 09 12:28:26.652114 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.652079    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="30df6f0a-26be-4d97-b1e0-d344a42c257e" containerName="manager"
Sep 09 12:28:26.652160 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.652146    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="30df6f0a-26be-4d97-b1e0-d344a42c257e" containerName="manager"
Sep 09 12:28:26.655269 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.655251    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"
Sep 09 12:28:26.671904 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.671879    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"]
Sep 09 12:28:26.746378 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.746350    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htwp9\" (UniqueName: \"kubernetes.io/projected/a6738b5d-a929-438b-bb05-4bfbd39f0c6e-kube-api-access-htwp9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wsvqn\" (UID: \"a6738b5d-a929-438b-bb05-4bfbd39f0c6e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"
Sep 09 12:28:26.746501 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.746414    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6738b5d-a929-438b-bb05-4bfbd39f0c6e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wsvqn\" (UID: \"a6738b5d-a929-438b-bb05-4bfbd39f0c6e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"
Sep 09 12:28:26.847659 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.847632    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htwp9\" (UniqueName: \"kubernetes.io/projected/a6738b5d-a929-438b-bb05-4bfbd39f0c6e-kube-api-access-htwp9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wsvqn\" (UID: \"a6738b5d-a929-438b-bb05-4bfbd39f0c6e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"
Sep 09 12:28:26.847757 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.847669    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6738b5d-a929-438b-bb05-4bfbd39f0c6e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wsvqn\" (UID: \"a6738b5d-a929-438b-bb05-4bfbd39f0c6e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"
Sep 09 12:28:26.848082 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.848063    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6738b5d-a929-438b-bb05-4bfbd39f0c6e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wsvqn\" (UID: \"a6738b5d-a929-438b-bb05-4bfbd39f0c6e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"
Sep 09 12:28:26.871362 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.871339    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htwp9\" (UniqueName: \"kubernetes.io/projected/a6738b5d-a929-438b-bb05-4bfbd39f0c6e-kube-api-access-htwp9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wsvqn\" (UID: \"a6738b5d-a929-438b-bb05-4bfbd39f0c6e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"
Sep 09 12:28:26.965683 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:26.965659    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"
Sep 09 12:28:27.251240 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:27.251215    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"]
Sep 09 12:28:27.253264 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:28:27.253236    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6738b5d_a929_438b_bb05_4bfbd39f0c6e.slice/crio-c630b1b6b038db3bb5b5fe0cae2828d241bea6862543026ec7f02f983fc4a1eb WatchSource:0}: Error finding container c630b1b6b038db3bb5b5fe0cae2828d241bea6862543026ec7f02f983fc4a1eb: Status 404 returned error can't find the container with id c630b1b6b038db3bb5b5fe0cae2828d241bea6862543026ec7f02f983fc4a1eb
Sep 09 12:28:27.255049 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:27.255015    2582 generic.go:356] "Generic (PLEG): container finished" podID="30df6f0a-26be-4d97-b1e0-d344a42c257e" containerID="a260015d29466141c5e15c034d6ffaacc740bc78e410f7b73b175001335774b2" exitCode=0
Sep 09 12:28:27.255133 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:27.255051    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"
Sep 09 12:28:27.255133 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:27.255094    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2" event={"ID":"30df6f0a-26be-4d97-b1e0-d344a42c257e","Type":"ContainerDied","Data":"a260015d29466141c5e15c034d6ffaacc740bc78e410f7b73b175001335774b2"}
Sep 09 12:28:27.255240 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:27.255142    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2" event={"ID":"30df6f0a-26be-4d97-b1e0-d344a42c257e","Type":"ContainerDied","Data":"25fb5996695ebf4d7a0bdbf2c680996e3c5d2628a6c6580cf948f9402a9ff51e"}
Sep 09 12:28:27.255240 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:27.255166    2582 scope.go:117] "RemoveContainer" containerID="a260015d29466141c5e15c034d6ffaacc740bc78e410f7b73b175001335774b2"
Sep 09 12:28:27.263866 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:27.263833    2582 scope.go:117] "RemoveContainer" containerID="a260015d29466141c5e15c034d6ffaacc740bc78e410f7b73b175001335774b2"
Sep 09 12:28:27.264123 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:27.264105    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a260015d29466141c5e15c034d6ffaacc740bc78e410f7b73b175001335774b2\": container with ID starting with a260015d29466141c5e15c034d6ffaacc740bc78e410f7b73b175001335774b2 not found: ID does not exist" containerID="a260015d29466141c5e15c034d6ffaacc740bc78e410f7b73b175001335774b2"
Sep 09 12:28:27.264181 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:27.264131    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a260015d29466141c5e15c034d6ffaacc740bc78e410f7b73b175001335774b2"} err="failed to get container status \"a260015d29466141c5e15c034d6ffaacc740bc78e410f7b73b175001335774b2\": rpc error: code = NotFound desc = could not find container \"a260015d29466141c5e15c034d6ffaacc740bc78e410f7b73b175001335774b2\": container with ID starting with a260015d29466141c5e15c034d6ffaacc740bc78e410f7b73b175001335774b2 not found: ID does not exist"
Sep 09 12:28:27.283316 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:27.283296    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"]
Sep 09 12:28:27.290728 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:27.290708    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9zhg2"]
Sep 09 12:28:28.263523 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:28.263481    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn" event={"ID":"a6738b5d-a929-438b-bb05-4bfbd39f0c6e","Type":"ContainerStarted","Data":"6eb71b6a83aff647efe5da5ab85f35f23eaa5f9fd1eb4709f98a6429d3e19c2c"}
Sep 09 12:28:28.264029 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:28.263547    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn" event={"ID":"a6738b5d-a929-438b-bb05-4bfbd39f0c6e","Type":"ContainerStarted","Data":"c630b1b6b038db3bb5b5fe0cae2828d241bea6862543026ec7f02f983fc4a1eb"}
Sep 09 12:28:28.264029 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:28.263575    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"
Sep 09 12:28:28.288384 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:28.288343    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn" podStartSLOduration=2.288327072 podStartE2EDuration="2.288327072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:28:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:28:28.287576245 +0000 UTC m=+527.984141388" watchObservedRunningTime="2026-09-09 12:28:28.288327072 +0000 UTC m=+527.984892221"
Sep 09 12:28:28.824701 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:28.824671    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30df6f0a-26be-4d97-b1e0-d344a42c257e" path="/var/lib/kubelet/pods/30df6f0a-26be-4d97-b1e0-d344a42c257e/volumes"
Sep 09 12:28:39.273215 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:39.273139    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"
Sep 09 12:28:39.324203 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:39.324174    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"]
Sep 09 12:28:39.324545 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:39.324374    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2" podUID="66445b0b-2779-4527-b53b-08dcedd60c95" containerName="manager" containerID="cri-o://ab9567c3b34215ac21abae4ce0b12296de2a6e3ef5c07ceb3f086439a1b267fa" gracePeriod=10
Sep 09 12:28:39.572213 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:39.572191    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"
Sep 09 12:28:39.650127 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:39.650095    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66445b0b-2779-4527-b53b-08dcedd60c95-extensions-socket-volume\") pod \"66445b0b-2779-4527-b53b-08dcedd60c95\" (UID: \"66445b0b-2779-4527-b53b-08dcedd60c95\") "
Sep 09 12:28:39.650306 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:39.650188    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tpd4\" (UniqueName: \"kubernetes.io/projected/66445b0b-2779-4527-b53b-08dcedd60c95-kube-api-access-6tpd4\") pod \"66445b0b-2779-4527-b53b-08dcedd60c95\" (UID: \"66445b0b-2779-4527-b53b-08dcedd60c95\") "
Sep 09 12:28:39.650472 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:39.650450    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66445b0b-2779-4527-b53b-08dcedd60c95-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "66445b0b-2779-4527-b53b-08dcedd60c95" (UID: "66445b0b-2779-4527-b53b-08dcedd60c95"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:28:39.652373 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:39.652349    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66445b0b-2779-4527-b53b-08dcedd60c95-kube-api-access-6tpd4" (OuterVolumeSpecName: "kube-api-access-6tpd4") pod "66445b0b-2779-4527-b53b-08dcedd60c95" (UID: "66445b0b-2779-4527-b53b-08dcedd60c95"). InnerVolumeSpecName "kube-api-access-6tpd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:28:39.751458 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:39.751427    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66445b0b-2779-4527-b53b-08dcedd60c95-extensions-socket-volume\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:28:39.751458 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:39.751451    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tpd4\" (UniqueName: \"kubernetes.io/projected/66445b0b-2779-4527-b53b-08dcedd60c95-kube-api-access-6tpd4\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:28:40.343519 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:40.343471    2582 generic.go:356] "Generic (PLEG): container finished" podID="66445b0b-2779-4527-b53b-08dcedd60c95" containerID="ab9567c3b34215ac21abae4ce0b12296de2a6e3ef5c07ceb3f086439a1b267fa" exitCode=0
Sep 09 12:28:40.343981 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:40.343519    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"
Sep 09 12:28:40.343981 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:40.343549    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2" event={"ID":"66445b0b-2779-4527-b53b-08dcedd60c95","Type":"ContainerDied","Data":"ab9567c3b34215ac21abae4ce0b12296de2a6e3ef5c07ceb3f086439a1b267fa"}
Sep 09 12:28:40.343981 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:40.343627    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2" event={"ID":"66445b0b-2779-4527-b53b-08dcedd60c95","Type":"ContainerDied","Data":"ebc906c500d31e0c1c39def99747c5041d7fd47aca79ff31169433a172d20600"}
Sep 09 12:28:40.343981 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:40.343653    2582 scope.go:117] "RemoveContainer" containerID="ab9567c3b34215ac21abae4ce0b12296de2a6e3ef5c07ceb3f086439a1b267fa"
Sep 09 12:28:40.354572 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:40.354552    2582 scope.go:117] "RemoveContainer" containerID="ab9567c3b34215ac21abae4ce0b12296de2a6e3ef5c07ceb3f086439a1b267fa"
Sep 09 12:28:40.355028 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:28:40.355000    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab9567c3b34215ac21abae4ce0b12296de2a6e3ef5c07ceb3f086439a1b267fa\": container with ID starting with ab9567c3b34215ac21abae4ce0b12296de2a6e3ef5c07ceb3f086439a1b267fa not found: ID does not exist" containerID="ab9567c3b34215ac21abae4ce0b12296de2a6e3ef5c07ceb3f086439a1b267fa"
Sep 09 12:28:40.355131 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:40.355044    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab9567c3b34215ac21abae4ce0b12296de2a6e3ef5c07ceb3f086439a1b267fa"} err="failed to get container status \"ab9567c3b34215ac21abae4ce0b12296de2a6e3ef5c07ceb3f086439a1b267fa\": rpc error: code = NotFound desc = could not find container \"ab9567c3b34215ac21abae4ce0b12296de2a6e3ef5c07ceb3f086439a1b267fa\": container with ID starting with ab9567c3b34215ac21abae4ce0b12296de2a6e3ef5c07ceb3f086439a1b267fa not found: ID does not exist"
Sep 09 12:28:40.366177 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:40.366152    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"]
Sep 09 12:28:40.372284 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:40.372261    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-nwpb2"]
Sep 09 12:28:40.825816 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:40.825791    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66445b0b-2779-4527-b53b-08dcedd60c95" path="/var/lib/kubelet/pods/66445b0b-2779-4527-b53b-08dcedd60c95/volumes"
Sep 09 12:28:42.668047 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.668015    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"]
Sep 09 12:28:42.668549 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.668534    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66445b0b-2779-4527-b53b-08dcedd60c95" containerName="manager"
Sep 09 12:28:42.668602 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.668553    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="66445b0b-2779-4527-b53b-08dcedd60c95" containerName="manager"
Sep 09 12:28:42.668671 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.668660    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="66445b0b-2779-4527-b53b-08dcedd60c95" containerName="manager"
Sep 09 12:28:42.672014 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.671990    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.674341 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.674319    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-gkj2g\""
Sep 09 12:28:42.674341 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.674330    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 12:28:42.681151 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.681127    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"]
Sep 09 12:28:42.775389 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.775361    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4efdd84f-7b2d-43a6-b028-5426f2b68203-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.775538 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.775408    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4efdd84f-7b2d-43a6-b028-5426f2b68203-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.775538 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.775431    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4efdd84f-7b2d-43a6-b028-5426f2b68203-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.775538 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.775453    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4efdd84f-7b2d-43a6-b028-5426f2b68203-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.775718 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.775532    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4efdd84f-7b2d-43a6-b028-5426f2b68203-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.775718 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.775611    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4efdd84f-7b2d-43a6-b028-5426f2b68203-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.775718 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.775643    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4efdd84f-7b2d-43a6-b028-5426f2b68203-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.775718 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.775672    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-984hg\" (UniqueName: \"kubernetes.io/projected/4efdd84f-7b2d-43a6-b028-5426f2b68203-kube-api-access-984hg\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.775718 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.775699    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4efdd84f-7b2d-43a6-b028-5426f2b68203-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.876475 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.876447    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4efdd84f-7b2d-43a6-b028-5426f2b68203-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.876475 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.876492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4efdd84f-7b2d-43a6-b028-5426f2b68203-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.876732 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.876533    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4efdd84f-7b2d-43a6-b028-5426f2b68203-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.876732 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.876572    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4efdd84f-7b2d-43a6-b028-5426f2b68203-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.876732 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.876650    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4efdd84f-7b2d-43a6-b028-5426f2b68203-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.876732 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.876678    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4efdd84f-7b2d-43a6-b028-5426f2b68203-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.876732 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.876704    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-984hg\" (UniqueName: \"kubernetes.io/projected/4efdd84f-7b2d-43a6-b028-5426f2b68203-kube-api-access-984hg\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.877042 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.876738    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4efdd84f-7b2d-43a6-b028-5426f2b68203-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.877042 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.876765    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4efdd84f-7b2d-43a6-b028-5426f2b68203-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.877042 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.876883    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4efdd84f-7b2d-43a6-b028-5426f2b68203-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.877206 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.877102    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4efdd84f-7b2d-43a6-b028-5426f2b68203-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.877700 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.877673    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4efdd84f-7b2d-43a6-b028-5426f2b68203-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.877865 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.877698    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4efdd84f-7b2d-43a6-b028-5426f2b68203-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.877865 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.877766    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4efdd84f-7b2d-43a6-b028-5426f2b68203-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.879617 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.879592    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4efdd84f-7b2d-43a6-b028-5426f2b68203-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.879832 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.879813    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4efdd84f-7b2d-43a6-b028-5426f2b68203-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.885218 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.885193    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-984hg\" (UniqueName: \"kubernetes.io/projected/4efdd84f-7b2d-43a6-b028-5426f2b68203-kube-api-access-984hg\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.885311 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.885227    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4efdd84f-7b2d-43a6-b028-5426f2b68203-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-msm2r\" (UID: \"4efdd84f-7b2d-43a6-b028-5426f2b68203\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:42.983562 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:42.983503    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:43.257162 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:43.257131    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"]
Sep 09 12:28:43.259730 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:28:43.259701    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4efdd84f_7b2d_43a6_b028_5426f2b68203.slice/crio-b458d6763a8966e30e9f573b80cf58f3519f0bfe772223bcde8dc7558299d8f8 WatchSource:0}: Error finding container b458d6763a8966e30e9f573b80cf58f3519f0bfe772223bcde8dc7558299d8f8: Status 404 returned error can't find the container with id b458d6763a8966e30e9f573b80cf58f3519f0bfe772223bcde8dc7558299d8f8
Sep 09 12:28:43.360926 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:43.360894    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r" event={"ID":"4efdd84f-7b2d-43a6-b028-5426f2b68203","Type":"ContainerStarted","Data":"b458d6763a8966e30e9f573b80cf58f3519f0bfe772223bcde8dc7558299d8f8"}
Sep 09 12:28:45.644613 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:45.644571    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:28:45.644918 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:45.644645    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:28:45.644918 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:45.644675    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:28:46.380637 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:46.380598    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r" event={"ID":"4efdd84f-7b2d-43a6-b028-5426f2b68203","Type":"ContainerStarted","Data":"5cc0998a2c68b2ef790c7a4bc7eda51d90748974ade47e0772acc70b22515f70"}
Sep 09 12:28:46.415353 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:46.415285    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r" podStartSLOduration=2.032942448 podStartE2EDuration="4.415264726s" totalImagesPullingTime="2.382322278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:28:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:28:46.412374836 +0000 UTC m=+546.108939990" watchObservedRunningTime="2026-09-09 12:28:46.415264726 +0000 UTC m=+546.111829880"
Sep 09 12:28:46.983916 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:46.983884    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:46.988834 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:46.988813    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:47.385806 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:47.385776    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:47.386794 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:47.386776    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:28:56.247988 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:56.247948    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-7kks7"]
Sep 09 12:28:56.251789 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:56.251766    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-7kks7"
Sep 09 12:28:56.253713 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:56.253697    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 09 12:28:56.253798 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:56.253737    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vb5hj\""
Sep 09 12:28:56.261319 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:56.261294    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-7kks7"]
Sep 09 12:28:56.343872 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:56.343823    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-7kks7"]
Sep 09 12:28:56.394343 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:56.394303    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkv2c\" (UniqueName: \"kubernetes.io/projected/749b1264-b4f5-4032-ac46-9beefa6f5ece-kube-api-access-lkv2c\") pod \"limitador-limitador-8588749977-7kks7\" (UID: \"749b1264-b4f5-4032-ac46-9beefa6f5ece\") " pod="kuadrant-system/limitador-limitador-8588749977-7kks7"
Sep 09 12:28:56.394518 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:56.394353    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/749b1264-b4f5-4032-ac46-9beefa6f5ece-config-file\") pod \"limitador-limitador-8588749977-7kks7\" (UID: \"749b1264-b4f5-4032-ac46-9beefa6f5ece\") " pod="kuadrant-system/limitador-limitador-8588749977-7kks7"
Sep 09 12:28:56.495127 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:56.495086    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkv2c\" (UniqueName: \"kubernetes.io/projected/749b1264-b4f5-4032-ac46-9beefa6f5ece-kube-api-access-lkv2c\") pod \"limitador-limitador-8588749977-7kks7\" (UID: \"749b1264-b4f5-4032-ac46-9beefa6f5ece\") " pod="kuadrant-system/limitador-limitador-8588749977-7kks7"
Sep 09 12:28:56.495307 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:56.495137    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/749b1264-b4f5-4032-ac46-9beefa6f5ece-config-file\") pod \"limitador-limitador-8588749977-7kks7\" (UID: \"749b1264-b4f5-4032-ac46-9beefa6f5ece\") " pod="kuadrant-system/limitador-limitador-8588749977-7kks7"
Sep 09 12:28:56.495837 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:56.495818    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/749b1264-b4f5-4032-ac46-9beefa6f5ece-config-file\") pod \"limitador-limitador-8588749977-7kks7\" (UID: \"749b1264-b4f5-4032-ac46-9beefa6f5ece\") " pod="kuadrant-system/limitador-limitador-8588749977-7kks7"
Sep 09 12:28:56.503836 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:56.503777    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkv2c\" (UniqueName: \"kubernetes.io/projected/749b1264-b4f5-4032-ac46-9beefa6f5ece-kube-api-access-lkv2c\") pod \"limitador-limitador-8588749977-7kks7\" (UID: \"749b1264-b4f5-4032-ac46-9beefa6f5ece\") " pod="kuadrant-system/limitador-limitador-8588749977-7kks7"
Sep 09 12:28:56.562553 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:56.562524    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-7kks7"
Sep 09 12:28:56.838595 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:56.838570    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-7kks7"]
Sep 09 12:28:56.840244 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:28:56.840218    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749b1264_b4f5_4032_ac46_9beefa6f5ece.slice/crio-ce3f64c3835bc53ff6bb42bac2f8e53e315a4891e47de2972d9e29faa895536b WatchSource:0}: Error finding container ce3f64c3835bc53ff6bb42bac2f8e53e315a4891e47de2972d9e29faa895536b: Status 404 returned error can't find the container with id ce3f64c3835bc53ff6bb42bac2f8e53e315a4891e47de2972d9e29faa895536b
Sep 09 12:28:57.086889 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:57.086840    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6bj8s"]
Sep 09 12:28:57.091296 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:57.091235    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-6bj8s"
Sep 09 12:28:57.093476 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:57.093455    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lsjjv\""
Sep 09 12:28:57.098740 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:57.098720    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6bj8s"]
Sep 09 12:28:57.201169 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:57.201134    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8c4j\" (UniqueName: \"kubernetes.io/projected/d69caab2-eec0-433b-bc36-ca99ebe09c3f-kube-api-access-r8c4j\") pod \"authorino-7bcfd8f9cd-6bj8s\" (UID: \"d69caab2-eec0-433b-bc36-ca99ebe09c3f\") " pod="kuadrant-system/authorino-7bcfd8f9cd-6bj8s"
Sep 09 12:28:57.302187 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:57.302153    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8c4j\" (UniqueName: \"kubernetes.io/projected/d69caab2-eec0-433b-bc36-ca99ebe09c3f-kube-api-access-r8c4j\") pod \"authorino-7bcfd8f9cd-6bj8s\" (UID: \"d69caab2-eec0-433b-bc36-ca99ebe09c3f\") " pod="kuadrant-system/authorino-7bcfd8f9cd-6bj8s"
Sep 09 12:28:57.310287 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:57.310262    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8c4j\" (UniqueName: \"kubernetes.io/projected/d69caab2-eec0-433b-bc36-ca99ebe09c3f-kube-api-access-r8c4j\") pod \"authorino-7bcfd8f9cd-6bj8s\" (UID: \"d69caab2-eec0-433b-bc36-ca99ebe09c3f\") " pod="kuadrant-system/authorino-7bcfd8f9cd-6bj8s"
Sep 09 12:28:57.402793 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:57.402728    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-6bj8s"
Sep 09 12:28:57.439599 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:57.439568    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-7kks7" event={"ID":"749b1264-b4f5-4032-ac46-9beefa6f5ece","Type":"ContainerStarted","Data":"ce3f64c3835bc53ff6bb42bac2f8e53e315a4891e47de2972d9e29faa895536b"}
Sep 09 12:28:57.675506 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:57.675449    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6bj8s"]
Sep 09 12:28:57.676569 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:28:57.676541    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd69caab2_eec0_433b_bc36_ca99ebe09c3f.slice/crio-c3220e4eff1073f5b31485eab5ac930c0fa71a45d96cd3187ef1ecce2d68974f WatchSource:0}: Error finding container c3220e4eff1073f5b31485eab5ac930c0fa71a45d96cd3187ef1ecce2d68974f: Status 404 returned error can't find the container with id c3220e4eff1073f5b31485eab5ac930c0fa71a45d96cd3187ef1ecce2d68974f
Sep 09 12:28:58.448563 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:58.448526    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-7kks7" event={"ID":"749b1264-b4f5-4032-ac46-9beefa6f5ece","Type":"ContainerStarted","Data":"714984d5bf431627bfbd4d74553b237afe6e7ab7ec881e09839531da27d873cd"}
Sep 09 12:28:58.449005 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:58.448624    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-7kks7"
Sep 09 12:28:58.449981 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:58.449952    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-6bj8s" event={"ID":"d69caab2-eec0-433b-bc36-ca99ebe09c3f","Type":"ContainerStarted","Data":"c3220e4eff1073f5b31485eab5ac930c0fa71a45d96cd3187ef1ecce2d68974f"}
Sep 09 12:28:58.467321 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:28:58.467278    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-7kks7" podStartSLOduration=1.329407078 podStartE2EDuration="2.467262097s" totalImagesPullingTime="1.137855019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:28:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:28:58.464788169 +0000 UTC m=+558.161353326" watchObservedRunningTime="2026-09-09 12:28:58.467262097 +0000 UTC m=+558.163827251"
Sep 09 12:29:00.466526 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:00.466484    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-6bj8s" event={"ID":"d69caab2-eec0-433b-bc36-ca99ebe09c3f","Type":"ContainerStarted","Data":"e6013babe2ec52824d7ec515e496216cf98bb6eaf76ad7aa638a7427aef09440"}
Sep 09 12:29:00.482646 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:00.482594    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-6bj8s" podStartSLOduration=1.747631706 podStartE2EDuration="3.482577353s" totalImagesPullingTime="1.734945647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:28:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:29:00.48095483 +0000 UTC m=+560.177520006" watchObservedRunningTime="2026-09-09 12:29:00.482577353 +0000 UTC m=+560.179142505"
Sep 09 12:29:00.959862 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:00.959812    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6bj8s"]
Sep 09 12:29:02.476835 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:02.476796    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-6bj8s" podUID="d69caab2-eec0-433b-bc36-ca99ebe09c3f" containerName="authorino" containerID="cri-o://e6013babe2ec52824d7ec515e496216cf98bb6eaf76ad7aa638a7427aef09440" gracePeriod=30
Sep 09 12:29:02.718569 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:02.718545    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-6bj8s"
Sep 09 12:29:02.851043 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:02.851014    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8c4j\" (UniqueName: \"kubernetes.io/projected/d69caab2-eec0-433b-bc36-ca99ebe09c3f-kube-api-access-r8c4j\") pod \"d69caab2-eec0-433b-bc36-ca99ebe09c3f\" (UID: \"d69caab2-eec0-433b-bc36-ca99ebe09c3f\") "
Sep 09 12:29:02.853156 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:02.853129    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d69caab2-eec0-433b-bc36-ca99ebe09c3f-kube-api-access-r8c4j" (OuterVolumeSpecName: "kube-api-access-r8c4j") pod "d69caab2-eec0-433b-bc36-ca99ebe09c3f" (UID: "d69caab2-eec0-433b-bc36-ca99ebe09c3f"). InnerVolumeSpecName "kube-api-access-r8c4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:29:02.952588 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:02.952558    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8c4j\" (UniqueName: \"kubernetes.io/projected/d69caab2-eec0-433b-bc36-ca99ebe09c3f-kube-api-access-r8c4j\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:29:03.483123 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:03.483083    2582 generic.go:356] "Generic (PLEG): container finished" podID="d69caab2-eec0-433b-bc36-ca99ebe09c3f" containerID="e6013babe2ec52824d7ec515e496216cf98bb6eaf76ad7aa638a7427aef09440" exitCode=0
Sep 09 12:29:03.483558 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:03.483130    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-6bj8s"
Sep 09 12:29:03.483558 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:03.483140    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-6bj8s" event={"ID":"d69caab2-eec0-433b-bc36-ca99ebe09c3f","Type":"ContainerDied","Data":"e6013babe2ec52824d7ec515e496216cf98bb6eaf76ad7aa638a7427aef09440"}
Sep 09 12:29:03.483558 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:03.483183    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-6bj8s" event={"ID":"d69caab2-eec0-433b-bc36-ca99ebe09c3f","Type":"ContainerDied","Data":"c3220e4eff1073f5b31485eab5ac930c0fa71a45d96cd3187ef1ecce2d68974f"}
Sep 09 12:29:03.483558 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:03.483200    2582 scope.go:117] "RemoveContainer" containerID="e6013babe2ec52824d7ec515e496216cf98bb6eaf76ad7aa638a7427aef09440"
Sep 09 12:29:03.492693 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:03.492675    2582 scope.go:117] "RemoveContainer" containerID="e6013babe2ec52824d7ec515e496216cf98bb6eaf76ad7aa638a7427aef09440"
Sep 09 12:29:03.492975 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:29:03.492956    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6013babe2ec52824d7ec515e496216cf98bb6eaf76ad7aa638a7427aef09440\": container with ID starting with e6013babe2ec52824d7ec515e496216cf98bb6eaf76ad7aa638a7427aef09440 not found: ID does not exist" containerID="e6013babe2ec52824d7ec515e496216cf98bb6eaf76ad7aa638a7427aef09440"
Sep 09 12:29:03.493036 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:03.492983    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6013babe2ec52824d7ec515e496216cf98bb6eaf76ad7aa638a7427aef09440"} err="failed to get container status \"e6013babe2ec52824d7ec515e496216cf98bb6eaf76ad7aa638a7427aef09440\": rpc error: code = NotFound desc = could not find container \"e6013babe2ec52824d7ec515e496216cf98bb6eaf76ad7aa638a7427aef09440\": container with ID starting with e6013babe2ec52824d7ec515e496216cf98bb6eaf76ad7aa638a7427aef09440 not found: ID does not exist"
Sep 09 12:29:03.508742 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:03.508711    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6bj8s"]
Sep 09 12:29:03.511342 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:03.511322    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6bj8s"]
Sep 09 12:29:04.824897 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:04.824845    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d69caab2-eec0-433b-bc36-ca99ebe09c3f" path="/var/lib/kubelet/pods/d69caab2-eec0-433b-bc36-ca99ebe09c3f/volumes"
Sep 09 12:29:09.460619 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:09.460584    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-7kks7"
Sep 09 12:29:11.952915 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:11.952883    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-7kks7"]
Sep 09 12:29:11.953549 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:11.953091    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-7kks7" podUID="749b1264-b4f5-4032-ac46-9beefa6f5ece" containerName="limitador" containerID="cri-o://714984d5bf431627bfbd4d74553b237afe6e7ab7ec881e09839531da27d873cd" gracePeriod=30
Sep 09 12:29:12.508484 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.508454    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-7kks7"
Sep 09 12:29:12.536235 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.536143    2582 generic.go:356] "Generic (PLEG): container finished" podID="749b1264-b4f5-4032-ac46-9beefa6f5ece" containerID="714984d5bf431627bfbd4d74553b237afe6e7ab7ec881e09839531da27d873cd" exitCode=0
Sep 09 12:29:12.536235 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.536188    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-7kks7"
Sep 09 12:29:12.536459 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.536230    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-7kks7" event={"ID":"749b1264-b4f5-4032-ac46-9beefa6f5ece","Type":"ContainerDied","Data":"714984d5bf431627bfbd4d74553b237afe6e7ab7ec881e09839531da27d873cd"}
Sep 09 12:29:12.536459 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.536287    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-7kks7" event={"ID":"749b1264-b4f5-4032-ac46-9beefa6f5ece","Type":"ContainerDied","Data":"ce3f64c3835bc53ff6bb42bac2f8e53e315a4891e47de2972d9e29faa895536b"}
Sep 09 12:29:12.536459 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.536318    2582 scope.go:117] "RemoveContainer" containerID="714984d5bf431627bfbd4d74553b237afe6e7ab7ec881e09839531da27d873cd"
Sep 09 12:29:12.545286 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.545268    2582 scope.go:117] "RemoveContainer" containerID="714984d5bf431627bfbd4d74553b237afe6e7ab7ec881e09839531da27d873cd"
Sep 09 12:29:12.545657 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:29:12.545637    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"714984d5bf431627bfbd4d74553b237afe6e7ab7ec881e09839531da27d873cd\": container with ID starting with 714984d5bf431627bfbd4d74553b237afe6e7ab7ec881e09839531da27d873cd not found: ID does not exist" containerID="714984d5bf431627bfbd4d74553b237afe6e7ab7ec881e09839531da27d873cd"
Sep 09 12:29:12.545734 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.545667    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"714984d5bf431627bfbd4d74553b237afe6e7ab7ec881e09839531da27d873cd"} err="failed to get container status \"714984d5bf431627bfbd4d74553b237afe6e7ab7ec881e09839531da27d873cd\": rpc error: code = NotFound desc = could not find container \"714984d5bf431627bfbd4d74553b237afe6e7ab7ec881e09839531da27d873cd\": container with ID starting with 714984d5bf431627bfbd4d74553b237afe6e7ab7ec881e09839531da27d873cd not found: ID does not exist"
Sep 09 12:29:12.629887 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.629842    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkv2c\" (UniqueName: \"kubernetes.io/projected/749b1264-b4f5-4032-ac46-9beefa6f5ece-kube-api-access-lkv2c\") pod \"749b1264-b4f5-4032-ac46-9beefa6f5ece\" (UID: \"749b1264-b4f5-4032-ac46-9beefa6f5ece\") "
Sep 09 12:29:12.630087 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.629926    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/749b1264-b4f5-4032-ac46-9beefa6f5ece-config-file\") pod \"749b1264-b4f5-4032-ac46-9beefa6f5ece\" (UID: \"749b1264-b4f5-4032-ac46-9beefa6f5ece\") "
Sep 09 12:29:12.630306 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.630283    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/749b1264-b4f5-4032-ac46-9beefa6f5ece-config-file" (OuterVolumeSpecName: "config-file") pod "749b1264-b4f5-4032-ac46-9beefa6f5ece" (UID: "749b1264-b4f5-4032-ac46-9beefa6f5ece"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:29:12.632402 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.632377    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/749b1264-b4f5-4032-ac46-9beefa6f5ece-kube-api-access-lkv2c" (OuterVolumeSpecName: "kube-api-access-lkv2c") pod "749b1264-b4f5-4032-ac46-9beefa6f5ece" (UID: "749b1264-b4f5-4032-ac46-9beefa6f5ece"). InnerVolumeSpecName "kube-api-access-lkv2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:29:12.731276 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.731231    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkv2c\" (UniqueName: \"kubernetes.io/projected/749b1264-b4f5-4032-ac46-9beefa6f5ece-kube-api-access-lkv2c\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:29:12.731276 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.731263    2582 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/749b1264-b4f5-4032-ac46-9beefa6f5ece-config-file\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:29:12.852682 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.852643    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-7kks7"]
Sep 09 12:29:12.856977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:12.856946    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-7kks7"]
Sep 09 12:29:14.393526 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.393495    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-l77jd"]
Sep 09 12:29:14.394151 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.394132    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="749b1264-b4f5-4032-ac46-9beefa6f5ece" containerName="limitador"
Sep 09 12:29:14.394208 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.394157    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="749b1264-b4f5-4032-ac46-9beefa6f5ece" containerName="limitador"
Sep 09 12:29:14.394208 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.394168    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d69caab2-eec0-433b-bc36-ca99ebe09c3f" containerName="authorino"
Sep 09 12:29:14.394208 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.394177    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d69caab2-eec0-433b-bc36-ca99ebe09c3f" containerName="authorino"
Sep 09 12:29:14.394303 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.394261    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d69caab2-eec0-433b-bc36-ca99ebe09c3f" containerName="authorino"
Sep 09 12:29:14.394303 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.394273    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="749b1264-b4f5-4032-ac46-9beefa6f5ece" containerName="limitador"
Sep 09 12:29:14.400316 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.400250    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l77jd"
Sep 09 12:29:14.402315 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.402287    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-4f99v\""
Sep 09 12:29:14.402452 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.402288    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:29:14.402970 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.402950    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 12:29:14.403119 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.402998    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 09 12:29:14.403678 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.403652    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-l77jd"]
Sep 09 12:29:14.548200 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.548173    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74xw2\" (UniqueName: \"kubernetes.io/projected/d761a477-dc03-4e3c-8093-26e1053df8f7-kube-api-access-74xw2\") pod \"postgres-69fb4594cf-l77jd\" (UID: \"d761a477-dc03-4e3c-8093-26e1053df8f7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l77jd"
Sep 09 12:29:14.548400 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.548210    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d761a477-dc03-4e3c-8093-26e1053df8f7-data\") pod \"postgres-69fb4594cf-l77jd\" (UID: \"d761a477-dc03-4e3c-8093-26e1053df8f7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l77jd"
Sep 09 12:29:14.649653 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.649571    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74xw2\" (UniqueName: \"kubernetes.io/projected/d761a477-dc03-4e3c-8093-26e1053df8f7-kube-api-access-74xw2\") pod \"postgres-69fb4594cf-l77jd\" (UID: \"d761a477-dc03-4e3c-8093-26e1053df8f7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l77jd"
Sep 09 12:29:14.649653 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.649609    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d761a477-dc03-4e3c-8093-26e1053df8f7-data\") pod \"postgres-69fb4594cf-l77jd\" (UID: \"d761a477-dc03-4e3c-8093-26e1053df8f7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l77jd"
Sep 09 12:29:14.649990 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.649972    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d761a477-dc03-4e3c-8093-26e1053df8f7-data\") pod \"postgres-69fb4594cf-l77jd\" (UID: \"d761a477-dc03-4e3c-8093-26e1053df8f7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l77jd"
Sep 09 12:29:14.659157 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.659134    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74xw2\" (UniqueName: \"kubernetes.io/projected/d761a477-dc03-4e3c-8093-26e1053df8f7-kube-api-access-74xw2\") pod \"postgres-69fb4594cf-l77jd\" (UID: \"d761a477-dc03-4e3c-8093-26e1053df8f7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l77jd"
Sep 09 12:29:14.712096 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.712059    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l77jd"
Sep 09 12:29:14.824338 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.824298    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="749b1264-b4f5-4032-ac46-9beefa6f5ece" path="/var/lib/kubelet/pods/749b1264-b4f5-4032-ac46-9beefa6f5ece/volumes"
Sep 09 12:29:14.984421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:14.984392    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-l77jd"]
Sep 09 12:29:14.985863 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:29:14.985827    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd761a477_dc03_4e3c_8093_26e1053df8f7.slice/crio-8bb7141bfd0ae60d3fd09a7c476bf153252f6c75db65b400a2e4ca1a80a998c6 WatchSource:0}: Error finding container 8bb7141bfd0ae60d3fd09a7c476bf153252f6c75db65b400a2e4ca1a80a998c6: Status 404 returned error can't find the container with id 8bb7141bfd0ae60d3fd09a7c476bf153252f6c75db65b400a2e4ca1a80a998c6
Sep 09 12:29:15.554959 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:15.554925    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l77jd" event={"ID":"d761a477-dc03-4e3c-8093-26e1053df8f7","Type":"ContainerStarted","Data":"8bb7141bfd0ae60d3fd09a7c476bf153252f6c75db65b400a2e4ca1a80a998c6"}
Sep 09 12:29:20.587591 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:20.587551    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l77jd" event={"ID":"d761a477-dc03-4e3c-8093-26e1053df8f7","Type":"ContainerStarted","Data":"7efb900db4d90f8cba87d13ac765596fc7cba65cc13e96e681dd444ad5c6cec2"}
Sep 09 12:29:20.588133 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:20.587645    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l77jd"
Sep 09 12:29:20.605156 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:20.605091    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l77jd" podStartSLOduration=1.561569701 podStartE2EDuration="6.605072004s" totalImagesPullingTime="5.043502303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:29:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:29:20.601797531 +0000 UTC m=+580.298362682" watchObservedRunningTime="2026-09-09 12:29:20.605072004 +0000 UTC m=+580.301637158"
Sep 09 12:29:26.623908 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:26.623870    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l77jd"
Sep 09 12:29:27.223082 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.223052    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-n6mlr"]
Sep 09 12:29:27.226640 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.226621    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-n6mlr"
Sep 09 12:29:27.228623 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.228602    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lsjjv\""
Sep 09 12:29:27.234961 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.234935    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-n6mlr"]
Sep 09 12:29:27.258726 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.258696    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzxtv\" (UniqueName: \"kubernetes.io/projected/78a782db-4572-4d30-8dd2-986c8245df02-kube-api-access-rzxtv\") pod \"authorino-557f876f69-n6mlr\" (UID: \"78a782db-4572-4d30-8dd2-986c8245df02\") " pod="kuadrant-system/authorino-557f876f69-n6mlr"
Sep 09 12:29:27.339178 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.339147    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-n6mlr"]
Sep 09 12:29:27.339436 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:29:27.339416    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rzxtv], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-n6mlr" podUID="78a782db-4572-4d30-8dd2-986c8245df02"
Sep 09 12:29:27.359403 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.359358    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzxtv\" (UniqueName: \"kubernetes.io/projected/78a782db-4572-4d30-8dd2-986c8245df02-kube-api-access-rzxtv\") pod \"authorino-557f876f69-n6mlr\" (UID: \"78a782db-4572-4d30-8dd2-986c8245df02\") " pod="kuadrant-system/authorino-557f876f69-n6mlr"
Sep 09 12:29:27.364346 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.364316    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-wvz28"]
Sep 09 12:29:27.367840 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.367812    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-wvz28"
Sep 09 12:29:27.369820 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.369797    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 09 12:29:27.371782 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.371758    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzxtv\" (UniqueName: \"kubernetes.io/projected/78a782db-4572-4d30-8dd2-986c8245df02-kube-api-access-rzxtv\") pod \"authorino-557f876f69-n6mlr\" (UID: \"78a782db-4572-4d30-8dd2-986c8245df02\") " pod="kuadrant-system/authorino-557f876f69-n6mlr"
Sep 09 12:29:27.377214 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.377189    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-wvz28"]
Sep 09 12:29:27.457197 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.457165    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-wvz28"]
Sep 09 12:29:27.457449 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:29:27.457430    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-j8r2d tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-j8r2d tls-cert]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-wvz28" podUID="e1ba4811-84bc-4dc4-8a96-782194a118ee"
Sep 09 12:29:27.460044 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.460014    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8r2d\" (UniqueName: \"kubernetes.io/projected/e1ba4811-84bc-4dc4-8a96-782194a118ee-kube-api-access-j8r2d\") pod \"authorino-6d4dfc649f-wvz28\" (UID: \"e1ba4811-84bc-4dc4-8a96-782194a118ee\") " pod="kuadrant-system/authorino-6d4dfc649f-wvz28"
Sep 09 12:29:27.460153 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.460063    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e1ba4811-84bc-4dc4-8a96-782194a118ee-tls-cert\") pod \"authorino-6d4dfc649f-wvz28\" (UID: \"e1ba4811-84bc-4dc4-8a96-782194a118ee\") " pod="kuadrant-system/authorino-6d4dfc649f-wvz28"
Sep 09 12:29:27.560813 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.560775    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8r2d\" (UniqueName: \"kubernetes.io/projected/e1ba4811-84bc-4dc4-8a96-782194a118ee-kube-api-access-j8r2d\") pod \"authorino-6d4dfc649f-wvz28\" (UID: \"e1ba4811-84bc-4dc4-8a96-782194a118ee\") " pod="kuadrant-system/authorino-6d4dfc649f-wvz28"
Sep 09 12:29:27.561031 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.560823    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e1ba4811-84bc-4dc4-8a96-782194a118ee-tls-cert\") pod \"authorino-6d4dfc649f-wvz28\" (UID: \"e1ba4811-84bc-4dc4-8a96-782194a118ee\") " pod="kuadrant-system/authorino-6d4dfc649f-wvz28"
Sep 09 12:29:27.563465 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.563432    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e1ba4811-84bc-4dc4-8a96-782194a118ee-tls-cert\") pod \"authorino-6d4dfc649f-wvz28\" (UID: \"e1ba4811-84bc-4dc4-8a96-782194a118ee\") " pod="kuadrant-system/authorino-6d4dfc649f-wvz28"
Sep 09 12:29:27.571790 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.571761    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8r2d\" (UniqueName: \"kubernetes.io/projected/e1ba4811-84bc-4dc4-8a96-782194a118ee-kube-api-access-j8r2d\") pod \"authorino-6d4dfc649f-wvz28\" (UID: \"e1ba4811-84bc-4dc4-8a96-782194a118ee\") " pod="kuadrant-system/authorino-6d4dfc649f-wvz28"
Sep 09 12:29:27.625902 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.625866    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-n6mlr"
Sep 09 12:29:27.626296 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.625882    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-wvz28"
Sep 09 12:29:27.633498 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.633470    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-n6mlr"
Sep 09 12:29:27.636273 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.636253    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-wvz28"
Sep 09 12:29:27.661237 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.661196    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzxtv\" (UniqueName: \"kubernetes.io/projected/78a782db-4572-4d30-8dd2-986c8245df02-kube-api-access-rzxtv\") pod \"78a782db-4572-4d30-8dd2-986c8245df02\" (UID: \"78a782db-4572-4d30-8dd2-986c8245df02\") "
Sep 09 12:29:27.661237 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.661238    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8r2d\" (UniqueName: \"kubernetes.io/projected/e1ba4811-84bc-4dc4-8a96-782194a118ee-kube-api-access-j8r2d\") pod \"e1ba4811-84bc-4dc4-8a96-782194a118ee\" (UID: \"e1ba4811-84bc-4dc4-8a96-782194a118ee\") "
Sep 09 12:29:27.661484 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.661289    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e1ba4811-84bc-4dc4-8a96-782194a118ee-tls-cert\") pod \"e1ba4811-84bc-4dc4-8a96-782194a118ee\" (UID: \"e1ba4811-84bc-4dc4-8a96-782194a118ee\") "
Sep 09 12:29:27.663727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.663662    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1ba4811-84bc-4dc4-8a96-782194a118ee-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "e1ba4811-84bc-4dc4-8a96-782194a118ee" (UID: "e1ba4811-84bc-4dc4-8a96-782194a118ee"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:29:27.663727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.663662    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1ba4811-84bc-4dc4-8a96-782194a118ee-kube-api-access-j8r2d" (OuterVolumeSpecName: "kube-api-access-j8r2d") pod "e1ba4811-84bc-4dc4-8a96-782194a118ee" (UID: "e1ba4811-84bc-4dc4-8a96-782194a118ee"). InnerVolumeSpecName "kube-api-access-j8r2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:29:27.663955 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.663812    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78a782db-4572-4d30-8dd2-986c8245df02-kube-api-access-rzxtv" (OuterVolumeSpecName: "kube-api-access-rzxtv") pod "78a782db-4572-4d30-8dd2-986c8245df02" (UID: "78a782db-4572-4d30-8dd2-986c8245df02"). InnerVolumeSpecName "kube-api-access-rzxtv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:29:27.762333 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.762294    2582 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e1ba4811-84bc-4dc4-8a96-782194a118ee-tls-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:29:27.762333 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.762328    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzxtv\" (UniqueName: \"kubernetes.io/projected/78a782db-4572-4d30-8dd2-986c8245df02-kube-api-access-rzxtv\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:29:27.762333 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:27.762339    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8r2d\" (UniqueName: \"kubernetes.io/projected/e1ba4811-84bc-4dc4-8a96-782194a118ee-kube-api-access-j8r2d\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:29:28.630645 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:28.630617    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-n6mlr"
Sep 09 12:29:28.631093 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:28.630618    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-wvz28"
Sep 09 12:29:28.661478 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:28.661448    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-n6mlr"]
Sep 09 12:29:28.665320 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:28.665290    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-n6mlr"]
Sep 09 12:29:28.686724 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:28.686688    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-wvz28"]
Sep 09 12:29:28.690680 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:28.690653    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-wvz28"]
Sep 09 12:29:28.824737 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:28.824701    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78a782db-4572-4d30-8dd2-986c8245df02" path="/var/lib/kubelet/pods/78a782db-4572-4d30-8dd2-986c8245df02/volumes"
Sep 09 12:29:28.824995 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:28.824982    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1ba4811-84bc-4dc4-8a96-782194a118ee" path="/var/lib/kubelet/pods/e1ba4811-84bc-4dc4-8a96-782194a118ee/volumes"
Sep 09 12:29:34.006590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.006551    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d7f9bfbfc-2s8xx"]
Sep 09 12:29:34.011038 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.011019    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx"
Sep 09 12:29:34.013231 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.013205    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 12:29:34.013372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.013215    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 12:29:34.013924 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.013897    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 12:29:34.014047 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.013900    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m7bgd\""
Sep 09 12:29:34.021907 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.021882    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d7f9bfbfc-2s8xx"]
Sep 09 12:29:34.105539 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.105503    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acd72f1e-0de2-41c7-b608-72e23e5823f5-metrics-certs\") pod \"maas-controller-6d7f9bfbfc-2s8xx\" (UID: \"acd72f1e-0de2-41c7-b608-72e23e5823f5\") " pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx"
Sep 09 12:29:34.105721 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.105550    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9h8c\" (UniqueName: \"kubernetes.io/projected/acd72f1e-0de2-41c7-b608-72e23e5823f5-kube-api-access-q9h8c\") pod \"maas-controller-6d7f9bfbfc-2s8xx\" (UID: \"acd72f1e-0de2-41c7-b608-72e23e5823f5\") " pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx"
Sep 09 12:29:34.105721 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.105622    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/acd72f1e-0de2-41c7-b608-72e23e5823f5-webhook-cert\") pod \"maas-controller-6d7f9bfbfc-2s8xx\" (UID: \"acd72f1e-0de2-41c7-b608-72e23e5823f5\") " pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx"
Sep 09 12:29:34.182806 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.182770    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-78bcb7bc6c-5xrn6"]
Sep 09 12:29:34.186643 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.186618    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6"
Sep 09 12:29:34.197841 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.197811    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78bcb7bc6c-5xrn6"]
Sep 09 12:29:34.206206 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.206160    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/acd72f1e-0de2-41c7-b608-72e23e5823f5-webhook-cert\") pod \"maas-controller-6d7f9bfbfc-2s8xx\" (UID: \"acd72f1e-0de2-41c7-b608-72e23e5823f5\") " pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx"
Sep 09 12:29:34.206364 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.206230    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-webhook-cert\") pod \"maas-controller-78bcb7bc6c-5xrn6\" (UID: \"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3\") " pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6"
Sep 09 12:29:34.206364 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.206292    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpfsw\" (UniqueName: \"kubernetes.io/projected/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-kube-api-access-lpfsw\") pod \"maas-controller-78bcb7bc6c-5xrn6\" (UID: \"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3\") " pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6"
Sep 09 12:29:34.206364 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.206336    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acd72f1e-0de2-41c7-b608-72e23e5823f5-metrics-certs\") pod \"maas-controller-6d7f9bfbfc-2s8xx\" (UID: \"acd72f1e-0de2-41c7-b608-72e23e5823f5\") " pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx"
Sep 09 12:29:34.206537 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.206366    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-metrics-certs\") pod \"maas-controller-78bcb7bc6c-5xrn6\" (UID: \"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3\") " pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6"
Sep 09 12:29:34.206537 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.206425    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9h8c\" (UniqueName: \"kubernetes.io/projected/acd72f1e-0de2-41c7-b608-72e23e5823f5-kube-api-access-q9h8c\") pod \"maas-controller-6d7f9bfbfc-2s8xx\" (UID: \"acd72f1e-0de2-41c7-b608-72e23e5823f5\") " pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx"
Sep 09 12:29:34.209062 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.209036    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/acd72f1e-0de2-41c7-b608-72e23e5823f5-webhook-cert\") pod \"maas-controller-6d7f9bfbfc-2s8xx\" (UID: \"acd72f1e-0de2-41c7-b608-72e23e5823f5\") " pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx"
Sep 09 12:29:34.209192 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.209038    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acd72f1e-0de2-41c7-b608-72e23e5823f5-metrics-certs\") pod \"maas-controller-6d7f9bfbfc-2s8xx\" (UID: \"acd72f1e-0de2-41c7-b608-72e23e5823f5\") " pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx"
Sep 09 12:29:34.217494 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.217459    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9h8c\" (UniqueName: \"kubernetes.io/projected/acd72f1e-0de2-41c7-b608-72e23e5823f5-kube-api-access-q9h8c\") pod \"maas-controller-6d7f9bfbfc-2s8xx\" (UID: \"acd72f1e-0de2-41c7-b608-72e23e5823f5\") " pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx"
Sep 09 12:29:34.307458 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.307351    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-webhook-cert\") pod \"maas-controller-78bcb7bc6c-5xrn6\" (UID: \"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3\") " pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6"
Sep 09 12:29:34.307458 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.307412    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpfsw\" (UniqueName: \"kubernetes.io/projected/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-kube-api-access-lpfsw\") pod \"maas-controller-78bcb7bc6c-5xrn6\" (UID: \"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3\") " pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6"
Sep 09 12:29:34.307458 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.307444    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-metrics-certs\") pod \"maas-controller-78bcb7bc6c-5xrn6\" (UID: \"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3\") " pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6"
Sep 09 12:29:34.309949 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.309924    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-metrics-certs\") pod \"maas-controller-78bcb7bc6c-5xrn6\" (UID: \"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3\") " pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6"
Sep 09 12:29:34.310076 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.310015    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-webhook-cert\") pod \"maas-controller-78bcb7bc6c-5xrn6\" (UID: \"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3\") " pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6"
Sep 09 12:29:34.315594 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.315568    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpfsw\" (UniqueName: \"kubernetes.io/projected/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-kube-api-access-lpfsw\") pod \"maas-controller-78bcb7bc6c-5xrn6\" (UID: \"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3\") " pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6"
Sep 09 12:29:34.323476 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.323451    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx"
Sep 09 12:29:34.497455 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.497407    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6"
Sep 09 12:29:34.596113 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.596084    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d7f9bfbfc-2s8xx"]
Sep 09 12:29:34.597623 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:29:34.597595    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd72f1e_0de2_41c7_b608_72e23e5823f5.slice/crio-d6389c1fc27232734959e9707b930713eeb16bd286efcf39fd4cb93e1c3de919 WatchSource:0}: Error finding container d6389c1fc27232734959e9707b930713eeb16bd286efcf39fd4cb93e1c3de919: Status 404 returned error can't find the container with id d6389c1fc27232734959e9707b930713eeb16bd286efcf39fd4cb93e1c3de919
Sep 09 12:29:34.669652 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.669615    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx" event={"ID":"acd72f1e-0de2-41c7-b608-72e23e5823f5","Type":"ContainerStarted","Data":"d6389c1fc27232734959e9707b930713eeb16bd286efcf39fd4cb93e1c3de919"}
Sep 09 12:29:34.772254 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:34.772230    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78bcb7bc6c-5xrn6"]
Sep 09 12:29:34.774045 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:29:34.774019    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c8fb8bd_97f3_441b_ac71_6a3efc7054e3.slice/crio-4ea1eb1fb7438306f41c06785dcc628d01a1dda5cad414ed2d87afe0f0d8176a WatchSource:0}: Error finding container 4ea1eb1fb7438306f41c06785dcc628d01a1dda5cad414ed2d87afe0f0d8176a: Status 404 returned error can't find the container with id 4ea1eb1fb7438306f41c06785dcc628d01a1dda5cad414ed2d87afe0f0d8176a
Sep 09 12:29:35.687502 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:35.687424    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6" event={"ID":"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3","Type":"ContainerStarted","Data":"4ea1eb1fb7438306f41c06785dcc628d01a1dda5cad414ed2d87afe0f0d8176a"}
Sep 09 12:29:37.705599 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:37.705489    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx" event={"ID":"acd72f1e-0de2-41c7-b608-72e23e5823f5","Type":"ContainerStarted","Data":"51d729e950a14e12a70c4c4e0a9e9db9551590e6f15a7221297f5529b37ea00a"}
Sep 09 12:29:37.706081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:37.705598    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx"
Sep 09 12:29:37.707294 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:37.707271    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6" event={"ID":"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3","Type":"ContainerStarted","Data":"54db88236d76f3b233c8d83a57600081222cea5df42789fb74e4aee9a05586af"}
Sep 09 12:29:37.707426 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:37.707331    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6"
Sep 09 12:29:37.724390 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:37.724338    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx" podStartSLOduration=2.020039375 podStartE2EDuration="4.724322714s" totalImagesPullingTime="2.704283339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:29:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:29:37.722726542 +0000 UTC m=+597.419291695" watchObservedRunningTime="2026-09-09 12:29:37.724322714 +0000 UTC m=+597.420887865"
Sep 09 12:29:37.741812 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:37.741753    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6" podStartSLOduration=1.203142331 podStartE2EDuration="3.741735827s" totalImagesPullingTime="2.538593496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:29:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:29:37.740368536 +0000 UTC m=+597.436933689" watchObservedRunningTime="2026-09-09 12:29:37.741735827 +0000 UTC m=+597.438301014"
Sep 09 12:29:48.714893 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:48.714844    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6"
Sep 09 12:29:48.717201 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:48.714939    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx"
Sep 09 12:29:48.779875 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:48.779818    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d7f9bfbfc-2s8xx"]
Sep 09 12:29:48.780076 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:48.780030    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx" podUID="acd72f1e-0de2-41c7-b608-72e23e5823f5" containerName="manager" containerID="cri-o://51d729e950a14e12a70c4c4e0a9e9db9551590e6f15a7221297f5529b37ea00a" gracePeriod=30
Sep 09 12:29:49.029578 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.029557    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx"
Sep 09 12:29:49.133660 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.133631    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/acd72f1e-0de2-41c7-b608-72e23e5823f5-webhook-cert\") pod \"acd72f1e-0de2-41c7-b608-72e23e5823f5\" (UID: \"acd72f1e-0de2-41c7-b608-72e23e5823f5\") "
Sep 09 12:29:49.133817 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.133677    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acd72f1e-0de2-41c7-b608-72e23e5823f5-metrics-certs\") pod \"acd72f1e-0de2-41c7-b608-72e23e5823f5\" (UID: \"acd72f1e-0de2-41c7-b608-72e23e5823f5\") "
Sep 09 12:29:49.133817 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.133725    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9h8c\" (UniqueName: \"kubernetes.io/projected/acd72f1e-0de2-41c7-b608-72e23e5823f5-kube-api-access-q9h8c\") pod \"acd72f1e-0de2-41c7-b608-72e23e5823f5\" (UID: \"acd72f1e-0de2-41c7-b608-72e23e5823f5\") "
Sep 09 12:29:49.136117 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.136087    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acd72f1e-0de2-41c7-b608-72e23e5823f5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "acd72f1e-0de2-41c7-b608-72e23e5823f5" (UID: "acd72f1e-0de2-41c7-b608-72e23e5823f5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:29:49.136210 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.136188    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acd72f1e-0de2-41c7-b608-72e23e5823f5-kube-api-access-q9h8c" (OuterVolumeSpecName: "kube-api-access-q9h8c") pod "acd72f1e-0de2-41c7-b608-72e23e5823f5" (UID: "acd72f1e-0de2-41c7-b608-72e23e5823f5"). InnerVolumeSpecName "kube-api-access-q9h8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:29:49.136253 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.136238    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acd72f1e-0de2-41c7-b608-72e23e5823f5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "acd72f1e-0de2-41c7-b608-72e23e5823f5" (UID: "acd72f1e-0de2-41c7-b608-72e23e5823f5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:29:49.234708 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.234680    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acd72f1e-0de2-41c7-b608-72e23e5823f5-metrics-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:29:49.234708 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.234706    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9h8c\" (UniqueName: \"kubernetes.io/projected/acd72f1e-0de2-41c7-b608-72e23e5823f5-kube-api-access-q9h8c\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:29:49.234879 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.234720    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/acd72f1e-0de2-41c7-b608-72e23e5823f5-webhook-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:29:49.776724 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.776690    2582 generic.go:356] "Generic (PLEG): container finished" podID="acd72f1e-0de2-41c7-b608-72e23e5823f5" containerID="51d729e950a14e12a70c4c4e0a9e9db9551590e6f15a7221297f5529b37ea00a" exitCode=0
Sep 09 12:29:49.777130 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.776739    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx"
Sep 09 12:29:49.777130 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.776742    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx" event={"ID":"acd72f1e-0de2-41c7-b608-72e23e5823f5","Type":"ContainerDied","Data":"51d729e950a14e12a70c4c4e0a9e9db9551590e6f15a7221297f5529b37ea00a"}
Sep 09 12:29:49.777130 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.776887    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d7f9bfbfc-2s8xx" event={"ID":"acd72f1e-0de2-41c7-b608-72e23e5823f5","Type":"ContainerDied","Data":"d6389c1fc27232734959e9707b930713eeb16bd286efcf39fd4cb93e1c3de919"}
Sep 09 12:29:49.777130 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.776924    2582 scope.go:117] "RemoveContainer" containerID="51d729e950a14e12a70c4c4e0a9e9db9551590e6f15a7221297f5529b37ea00a"
Sep 09 12:29:49.787004 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.786815    2582 scope.go:117] "RemoveContainer" containerID="51d729e950a14e12a70c4c4e0a9e9db9551590e6f15a7221297f5529b37ea00a"
Sep 09 12:29:49.787117 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:29:49.787094    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51d729e950a14e12a70c4c4e0a9e9db9551590e6f15a7221297f5529b37ea00a\": container with ID starting with 51d729e950a14e12a70c4c4e0a9e9db9551590e6f15a7221297f5529b37ea00a not found: ID does not exist" containerID="51d729e950a14e12a70c4c4e0a9e9db9551590e6f15a7221297f5529b37ea00a"
Sep 09 12:29:49.787174 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.787128    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51d729e950a14e12a70c4c4e0a9e9db9551590e6f15a7221297f5529b37ea00a"} err="failed to get container status \"51d729e950a14e12a70c4c4e0a9e9db9551590e6f15a7221297f5529b37ea00a\": rpc error: code = NotFound desc = could not find container \"51d729e950a14e12a70c4c4e0a9e9db9551590e6f15a7221297f5529b37ea00a\": container with ID starting with 51d729e950a14e12a70c4c4e0a9e9db9551590e6f15a7221297f5529b37ea00a not found: ID does not exist"
Sep 09 12:29:49.800109 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.800087    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d7f9bfbfc-2s8xx"]
Sep 09 12:29:49.804605 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:49.804584    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d7f9bfbfc-2s8xx"]
Sep 09 12:29:50.823986 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:29:50.823944    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acd72f1e-0de2-41c7-b608-72e23e5823f5" path="/var/lib/kubelet/pods/acd72f1e-0de2-41c7-b608-72e23e5823f5/volumes"
Sep 09 12:30:11.613860 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.613766    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"]
Sep 09 12:30:11.614279 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.614154    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acd72f1e-0de2-41c7-b608-72e23e5823f5" containerName="manager"
Sep 09 12:30:11.614279 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.614167    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="acd72f1e-0de2-41c7-b608-72e23e5823f5" containerName="manager"
Sep 09 12:30:11.614279 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.614236    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="acd72f1e-0de2-41c7-b608-72e23e5823f5" containerName="manager"
Sep 09 12:30:11.619676 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.619655    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:11.622182 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.622158    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 12:30:11.622292 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.622179    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5rdxz\""
Sep 09 12:30:11.622292 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.622158    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 09 12:30:11.628441 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.628412    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"]
Sep 09 12:30:11.707821 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.707788    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lztgt\" (UniqueName: \"kubernetes.io/projected/1451c9d8-ca35-4a61-b8a9-0912b4692982-kube-api-access-lztgt\") pod \"maas-api-59675b95bf-qg6b2\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") " pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:11.707997 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.707825    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1451c9d8-ca35-4a61-b8a9-0912b4692982-openshift-service-ca\") pod \"maas-api-59675b95bf-qg6b2\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") " pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:11.707997 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.707866    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1451c9d8-ca35-4a61-b8a9-0912b4692982-maas-api-tls\") pod \"maas-api-59675b95bf-qg6b2\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") " pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:11.707997 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.707957    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1451c9d8-ca35-4a61-b8a9-0912b4692982-metrics-certs\") pod \"maas-api-59675b95bf-qg6b2\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") " pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:11.809162 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.809132    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lztgt\" (UniqueName: \"kubernetes.io/projected/1451c9d8-ca35-4a61-b8a9-0912b4692982-kube-api-access-lztgt\") pod \"maas-api-59675b95bf-qg6b2\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") " pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:11.809162 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.809164    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1451c9d8-ca35-4a61-b8a9-0912b4692982-openshift-service-ca\") pod \"maas-api-59675b95bf-qg6b2\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") " pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:11.809376 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.809183    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1451c9d8-ca35-4a61-b8a9-0912b4692982-maas-api-tls\") pod \"maas-api-59675b95bf-qg6b2\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") " pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:11.809376 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.809222    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1451c9d8-ca35-4a61-b8a9-0912b4692982-metrics-certs\") pod \"maas-api-59675b95bf-qg6b2\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") " pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:11.810014 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.809929    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1451c9d8-ca35-4a61-b8a9-0912b4692982-openshift-service-ca\") pod \"maas-api-59675b95bf-qg6b2\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") " pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:11.811943 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.811907    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1451c9d8-ca35-4a61-b8a9-0912b4692982-metrics-certs\") pod \"maas-api-59675b95bf-qg6b2\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") " pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:11.812016 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.811939    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1451c9d8-ca35-4a61-b8a9-0912b4692982-maas-api-tls\") pod \"maas-api-59675b95bf-qg6b2\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") " pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:11.822397 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.822379    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lztgt\" (UniqueName: \"kubernetes.io/projected/1451c9d8-ca35-4a61-b8a9-0912b4692982-kube-api-access-lztgt\") pod \"maas-api-59675b95bf-qg6b2\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") " pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:11.931317 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:11.931236    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:12.191823 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.191749    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-hclrp"]
Sep 09 12:30:12.196259 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.196238    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-hclrp"
Sep 09 12:30:12.198317 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.198297    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 09 12:30:12.198411 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.198297    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-c8fzl\""
Sep 09 12:30:12.206719 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.206661    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-hclrp"]
Sep 09 12:30:12.210362 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.210344    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"]
Sep 09 12:30:12.211898 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:30:12.211870    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1451c9d8_ca35_4a61_b8a9_0912b4692982.slice/crio-972b1dcfd5133641e1c9cd9a238c7343826f391d8edabb0a351f44e09468e91f WatchSource:0}: Error finding container 972b1dcfd5133641e1c9cd9a238c7343826f391d8edabb0a351f44e09468e91f: Status 404 returned error can't find the container with id 972b1dcfd5133641e1c9cd9a238c7343826f391d8edabb0a351f44e09468e91f
Sep 09 12:30:12.213188 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.213173    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:30:12.313345 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.313298    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j899h\" (UniqueName: \"kubernetes.io/projected/fec7ad7e-69c0-4f61-9c45-4214164224b5-kube-api-access-j899h\") pod \"payload-processing-5b54b7fbb7-hclrp\" (UID: \"fec7ad7e-69c0-4f61-9c45-4214164224b5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hclrp"
Sep 09 12:30:12.313504 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.313376    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fec7ad7e-69c0-4f61-9c45-4214164224b5-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-hclrp\" (UID: \"fec7ad7e-69c0-4f61-9c45-4214164224b5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hclrp"
Sep 09 12:30:12.414033 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.414000    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j899h\" (UniqueName: \"kubernetes.io/projected/fec7ad7e-69c0-4f61-9c45-4214164224b5-kube-api-access-j899h\") pod \"payload-processing-5b54b7fbb7-hclrp\" (UID: \"fec7ad7e-69c0-4f61-9c45-4214164224b5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hclrp"
Sep 09 12:30:12.414199 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.414096    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fec7ad7e-69c0-4f61-9c45-4214164224b5-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-hclrp\" (UID: \"fec7ad7e-69c0-4f61-9c45-4214164224b5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hclrp"
Sep 09 12:30:12.415025 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.415000    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fec7ad7e-69c0-4f61-9c45-4214164224b5-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-hclrp\" (UID: \"fec7ad7e-69c0-4f61-9c45-4214164224b5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hclrp"
Sep 09 12:30:12.422715 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.422690    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j899h\" (UniqueName: \"kubernetes.io/projected/fec7ad7e-69c0-4f61-9c45-4214164224b5-kube-api-access-j899h\") pod \"payload-processing-5b54b7fbb7-hclrp\" (UID: \"fec7ad7e-69c0-4f61-9c45-4214164224b5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hclrp"
Sep 09 12:30:12.509191 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.509168    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-hclrp"
Sep 09 12:30:12.784266 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.784237    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-hclrp"]
Sep 09 12:30:12.785981 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:30:12.785956    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfec7ad7e_69c0_4f61_9c45_4214164224b5.slice/crio-db659d62cbbb3df604e353c0a67ddb9da8128e33391543b49ce16a50eeb2b9c9 WatchSource:0}: Error finding container db659d62cbbb3df604e353c0a67ddb9da8128e33391543b49ce16a50eeb2b9c9: Status 404 returned error can't find the container with id db659d62cbbb3df604e353c0a67ddb9da8128e33391543b49ce16a50eeb2b9c9
Sep 09 12:30:12.906417 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.906378    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-hclrp" event={"ID":"fec7ad7e-69c0-4f61-9c45-4214164224b5","Type":"ContainerStarted","Data":"db659d62cbbb3df604e353c0a67ddb9da8128e33391543b49ce16a50eeb2b9c9"}
Sep 09 12:30:12.907920 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:12.907883    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2" event={"ID":"1451c9d8-ca35-4a61-b8a9-0912b4692982","Type":"ContainerStarted","Data":"972b1dcfd5133641e1c9cd9a238c7343826f391d8edabb0a351f44e09468e91f"}
Sep 09 12:30:14.925935 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:14.925890    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2" event={"ID":"1451c9d8-ca35-4a61-b8a9-0912b4692982","Type":"ContainerStarted","Data":"7cf8e27e06e2571a98332deb2a01f10b47ffc49809f0df5649d92ae9a318b6f9"}
Sep 09 12:30:14.926385 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:14.926066    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:14.945003 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:14.944945    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2" podStartSLOduration=1.8223697620000001 podStartE2EDuration="3.944926871s" totalImagesPullingTime="2.122557109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:30:14.943276323 +0000 UTC m=+634.639841501" watchObservedRunningTime="2026-09-09 12:30:14.944926871 +0000 UTC m=+634.641492027"
Sep 09 12:30:15.933005 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:15.932966    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-hclrp" event={"ID":"fec7ad7e-69c0-4f61-9c45-4214164224b5","Type":"ContainerStarted","Data":"8713fc6130ffcd9dceccecace5826ec590bd8cde613537ae13ffdb7d76dc0040"}
Sep 09 12:30:15.933488 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:15.933085    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-hclrp"
Sep 09 12:30:15.952589 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:15.952536    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-hclrp" podStartSLOduration=1.7044207550000001 podStartE2EDuration="3.952520152s" totalImagesPullingTime="2.248099397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:30:15.952184768 +0000 UTC m=+635.648749933" watchObservedRunningTime="2026-09-09 12:30:15.952520152 +0000 UTC m=+635.649085303"
Sep 09 12:30:20.939696 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:20.939667    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:30:21.967663 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:21.967629    2582 generic.go:356] "Generic (PLEG): container finished" podID="4efdd84f-7b2d-43a6-b028-5426f2b68203" containerID="5cc0998a2c68b2ef790c7a4bc7eda51d90748974ade47e0772acc70b22515f70" exitCode=0
Sep 09 12:30:21.968125 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:21.967756    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r" event={"ID":"4efdd84f-7b2d-43a6-b028-5426f2b68203","Type":"ContainerDied","Data":"5cc0998a2c68b2ef790c7a4bc7eda51d90748974ade47e0772acc70b22515f70"}
Sep 09 12:30:21.968332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:21.968317    2582 scope.go:117] "RemoveContainer" containerID="5cc0998a2c68b2ef790c7a4bc7eda51d90748974ade47e0772acc70b22515f70"
Sep 09 12:30:21.969386 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:21.969359    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:30:21.969443 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:21.969425    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:30:21.969479 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:21.969457    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:30:22.979703 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:22.979659    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r" event={"ID":"4efdd84f-7b2d-43a6-b028-5426f2b68203","Type":"ContainerStarted","Data":"fb4485f1c3757eccb1b36170f624c02adb59673eef05ca7cc3ff38baaa6b24c2"}
Sep 09 12:30:22.983697 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:22.983665    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:30:23.989533 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:23.989496    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-msm2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 09 12:30:23.990075 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:23.989565    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r" podUID="4efdd84f-7b2d-43a6-b028-5426f2b68203" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 09 12:30:24.984958 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:24.984913    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-msm2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 09 12:30:24.985168 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:24.984982    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r" podUID="4efdd84f-7b2d-43a6-b028-5426f2b68203" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 09 12:30:25.983953 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:25.983919    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-msm2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 09 12:30:25.984532 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:25.984499    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r" podUID="4efdd84f-7b2d-43a6-b028-5426f2b68203" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 09 12:30:26.940407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:26.940378    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-hclrp"
Sep 09 12:30:26.988208 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:26.988175    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-msm2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 09 12:30:26.988741 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:26.988707    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r" podUID="4efdd84f-7b2d-43a6-b028-5426f2b68203" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 09 12:30:27.987881 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:27.987832    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:30:27.988241 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:27.988215    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:30:27.988911 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:27.988895    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-msm2r"
Sep 09 12:30:45.127951 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.127915    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"]
Sep 09 12:30:45.135312 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.135283    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.137480 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.137455    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2r6ws\""
Sep 09 12:30:45.137602 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.137498    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 09 12:30:45.137996 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.137978    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 09 12:30:45.138096 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.138031    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 09 12:30:45.155113 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.155091    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"]
Sep 09 12:30:45.303273 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.303237    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxqsl\" (UniqueName: \"kubernetes.io/projected/29ebf290-5def-4871-a5e7-6a17e4d3f836-kube-api-access-bxqsl\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.303273 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.303277    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29ebf290-5def-4871-a5e7-6a17e4d3f836-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.303515 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.303295    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29ebf290-5def-4871-a5e7-6a17e4d3f836-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.303515 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.303403    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29ebf290-5def-4871-a5e7-6a17e4d3f836-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.303515 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.303460    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29ebf290-5def-4871-a5e7-6a17e4d3f836-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.303515 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.303495    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29ebf290-5def-4871-a5e7-6a17e4d3f836-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.404067 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.403983    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxqsl\" (UniqueName: \"kubernetes.io/projected/29ebf290-5def-4871-a5e7-6a17e4d3f836-kube-api-access-bxqsl\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.404067 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.404032    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29ebf290-5def-4871-a5e7-6a17e4d3f836-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.404067 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.404058    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29ebf290-5def-4871-a5e7-6a17e4d3f836-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.404331 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.404099    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29ebf290-5def-4871-a5e7-6a17e4d3f836-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.404331 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.404143    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29ebf290-5def-4871-a5e7-6a17e4d3f836-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.404331 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.404171    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29ebf290-5def-4871-a5e7-6a17e4d3f836-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.404486 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.404424    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29ebf290-5def-4871-a5e7-6a17e4d3f836-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.404486 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.404442    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29ebf290-5def-4871-a5e7-6a17e4d3f836-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.404486 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.404469    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29ebf290-5def-4871-a5e7-6a17e4d3f836-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.406512 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.406483    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29ebf290-5def-4871-a5e7-6a17e4d3f836-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.406646 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.406631    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29ebf290-5def-4871-a5e7-6a17e4d3f836-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.412003 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.411984    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxqsl\" (UniqueName: \"kubernetes.io/projected/29ebf290-5def-4871-a5e7-6a17e4d3f836-kube-api-access-bxqsl\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-njrps\" (UID: \"29ebf290-5def-4871-a5e7-6a17e4d3f836\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.448057 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.448036    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:45.723449 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:45.723425    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"]
Sep 09 12:30:45.726060 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:30:45.726032    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ebf290_5def_4871_a5e7_6a17e4d3f836.slice/crio-af0104577b6f90a70ae6fb14a17305bfa1d1e48298a3d77a09e6ef34f1384e11 WatchSource:0}: Error finding container af0104577b6f90a70ae6fb14a17305bfa1d1e48298a3d77a09e6ef34f1384e11: Status 404 returned error can't find the container with id af0104577b6f90a70ae6fb14a17305bfa1d1e48298a3d77a09e6ef34f1384e11
Sep 09 12:30:46.111038 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:46.111004    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps" event={"ID":"29ebf290-5def-4871-a5e7-6a17e4d3f836","Type":"ContainerStarted","Data":"af0104577b6f90a70ae6fb14a17305bfa1d1e48298a3d77a09e6ef34f1384e11"}
Sep 09 12:30:47.118001 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:47.117921    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps" event={"ID":"29ebf290-5def-4871-a5e7-6a17e4d3f836","Type":"ContainerStarted","Data":"914b2c3b833dbfe32b44897adb19ea743a09cd82551b135039ac5f75121cd369"}
Sep 09 12:30:47.140844 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:47.140792    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps" podStartSLOduration=1.085444001 podStartE2EDuration="2.14077792s" totalImagesPullingTime="1.055333919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:30:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:30:47.139020279 +0000 UTC m=+666.835585431" watchObservedRunningTime="2026-09-09 12:30:47.14077792 +0000 UTC m=+666.837343071"
Sep 09 12:30:55.448939 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:55.448908    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:55.449371 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:55.448966    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:55.461218 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:55.461193    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:30:56.179760 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:30:56.179733    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-njrps"
Sep 09 12:32:50.696468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:50.696428    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-65c77c4788-b9pxf"]
Sep 09 12:32:50.700164 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:50.700131    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65c77c4788-b9pxf"
Sep 09 12:32:50.703364 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:50.703134    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lsjjv\""
Sep 09 12:32:50.703364 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:50.703234    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 09 12:32:50.706322 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:50.706297    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65c77c4788-b9pxf"]
Sep 09 12:32:50.772057 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:50.772033    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-867q7\" (UniqueName: \"kubernetes.io/projected/f42aff64-617a-4093-b460-a1be1194394a-kube-api-access-867q7\") pod \"authorino-65c77c4788-b9pxf\" (UID: \"f42aff64-617a-4093-b460-a1be1194394a\") " pod="kuadrant-system/authorino-65c77c4788-b9pxf"
Sep 09 12:32:50.772185 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:50.772138    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f42aff64-617a-4093-b460-a1be1194394a-tls-cert\") pod \"authorino-65c77c4788-b9pxf\" (UID: \"f42aff64-617a-4093-b460-a1be1194394a\") " pod="kuadrant-system/authorino-65c77c4788-b9pxf"
Sep 09 12:32:50.873478 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:50.873449    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f42aff64-617a-4093-b460-a1be1194394a-tls-cert\") pod \"authorino-65c77c4788-b9pxf\" (UID: \"f42aff64-617a-4093-b460-a1be1194394a\") " pod="kuadrant-system/authorino-65c77c4788-b9pxf"
Sep 09 12:32:50.873616 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:50.873509    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-867q7\" (UniqueName: \"kubernetes.io/projected/f42aff64-617a-4093-b460-a1be1194394a-kube-api-access-867q7\") pod \"authorino-65c77c4788-b9pxf\" (UID: \"f42aff64-617a-4093-b460-a1be1194394a\") " pod="kuadrant-system/authorino-65c77c4788-b9pxf"
Sep 09 12:32:50.876204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:50.876177    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f42aff64-617a-4093-b460-a1be1194394a-tls-cert\") pod \"authorino-65c77c4788-b9pxf\" (UID: \"f42aff64-617a-4093-b460-a1be1194394a\") " pod="kuadrant-system/authorino-65c77c4788-b9pxf"
Sep 09 12:32:50.883263 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:50.883219    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-867q7\" (UniqueName: \"kubernetes.io/projected/f42aff64-617a-4093-b460-a1be1194394a-kube-api-access-867q7\") pod \"authorino-65c77c4788-b9pxf\" (UID: \"f42aff64-617a-4093-b460-a1be1194394a\") " pod="kuadrant-system/authorino-65c77c4788-b9pxf"
Sep 09 12:32:51.012217 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:51.012198    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65c77c4788-b9pxf"
Sep 09 12:32:51.286992 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:32:51.286965    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf42aff64_617a_4093_b460_a1be1194394a.slice/crio-986ba25de033a577b2bd698c4f99b4096b683937e22e8cb639dffcc0048ceeca WatchSource:0}: Error finding container 986ba25de033a577b2bd698c4f99b4096b683937e22e8cb639dffcc0048ceeca: Status 404 returned error can't find the container with id 986ba25de033a577b2bd698c4f99b4096b683937e22e8cb639dffcc0048ceeca
Sep 09 12:32:51.289898 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:51.289865    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65c77c4788-b9pxf"]
Sep 09 12:32:51.804205 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:51.804167    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65c77c4788-b9pxf" event={"ID":"f42aff64-617a-4093-b460-a1be1194394a","Type":"ContainerStarted","Data":"986ba25de033a577b2bd698c4f99b4096b683937e22e8cb639dffcc0048ceeca"}
Sep 09 12:32:52.814867 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:52.814798    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65c77c4788-b9pxf" event={"ID":"f42aff64-617a-4093-b460-a1be1194394a","Type":"ContainerStarted","Data":"40c87e598b2f2828d4046a579a1fdaf7b31252f789d74e3e096d1b0d2533cc0d"}
Sep 09 12:32:52.845301 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:32:52.845244    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-65c77c4788-b9pxf" podStartSLOduration=1.8099523290000001 podStartE2EDuration="2.845226995s" totalImagesPullingTime="1.035274666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:32:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:32:52.842193216 +0000 UTC m=+792.538758369" watchObservedRunningTime="2026-09-09 12:32:52.845226995 +0000 UTC m=+792.541792146"
Sep 09 12:33:10.874616 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:10.874516    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"]
Sep 09 12:33:10.878965 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:10.878932    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:10.881458 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:10.881427    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-1e223f-openshift-default-dockercfg-29m28\""
Sep 09 12:33:10.891585 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:10.891554    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"]
Sep 09 12:33:10.959374 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:10.959327    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9g4b\" (UniqueName: \"kubernetes.io/projected/afcba06e-529f-471a-a7fb-9df792f78662-kube-api-access-p9g4b\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:10.959557 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:10.959386    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-credential-socket\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:10.959557 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:10.959501    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-workload-socket\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:10.959557 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:10.959544    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-workload-certs\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:10.959722 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:10.959570    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-istio-envoy\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:10.959722 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:10.959594    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afcba06e-529f-471a-a7fb-9df792f78662-istiod-ca-cert\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:10.959722 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:10.959642    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-istio-data\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:10.959722 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:10.959666    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afcba06e-529f-471a-a7fb-9df792f78662-istio-token\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:10.960372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:10.959760    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afcba06e-529f-471a-a7fb-9df792f78662-istio-podinfo\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.061030 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.060994    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-workload-socket\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.061030 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.061032    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-workload-certs\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.061269 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.061049    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-istio-envoy\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.061269 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.061190    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afcba06e-529f-471a-a7fb-9df792f78662-istiod-ca-cert\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.061269 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.061250    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-istio-data\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.061433 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.061277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afcba06e-529f-471a-a7fb-9df792f78662-istio-token\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.061433 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.061327    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afcba06e-529f-471a-a7fb-9df792f78662-istio-podinfo\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.061433 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.061408    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9g4b\" (UniqueName: \"kubernetes.io/projected/afcba06e-529f-471a-a7fb-9df792f78662-kube-api-access-p9g4b\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.061433 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.061416    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-workload-socket\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.061651 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.061450    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-workload-certs\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.061651 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.061455    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-credential-socket\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.061771 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.061683    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-istio-data\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.061771 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.061714    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-credential-socket\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.062057 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.062031    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afcba06e-529f-471a-a7fb-9df792f78662-istiod-ca-cert\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.063490 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.063460    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-istio-envoy\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.064245 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.064224    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afcba06e-529f-471a-a7fb-9df792f78662-istio-podinfo\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.080443 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.080412    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9g4b\" (UniqueName: \"kubernetes.io/projected/afcba06e-529f-471a-a7fb-9df792f78662-kube-api-access-p9g4b\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.081191 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.081172    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afcba06e-529f-471a-a7fb-9df792f78662-istio-token\") pod \"e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") " pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.194283 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.194193    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:11.481066 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.481028    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"]
Sep 09 12:33:11.483583 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:33:11.483546    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafcba06e_529f_471a_a7fb_9df792f78662.slice/crio-93d113d8da059e342a7936123910376c244266a7c6ed52c1e25d814ff4170e86 WatchSource:0}: Error finding container 93d113d8da059e342a7936123910376c244266a7c6ed52c1e25d814ff4170e86: Status 404 returned error can't find the container with id 93d113d8da059e342a7936123910376c244266a7c6ed52c1e25d814ff4170e86
Sep 09 12:33:11.486253 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.486209    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:33:11.486381 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.486302    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:33:11.486381 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.486344    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:33:11.928502 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.928465    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn" event={"ID":"afcba06e-529f-471a-a7fb-9df792f78662","Type":"ContainerStarted","Data":"bd44f442fadac3da25165cf740311b94cbeb35336826b153fca717cbac030d64"}
Sep 09 12:33:11.929012 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.928525    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn" event={"ID":"afcba06e-529f-471a-a7fb-9df792f78662","Type":"ContainerStarted","Data":"93d113d8da059e342a7936123910376c244266a7c6ed52c1e25d814ff4170e86"}
Sep 09 12:33:11.956321 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:11.956247    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn" podStartSLOduration=1.956224455 podStartE2EDuration="1.956224455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:33:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:33:11.952455159 +0000 UTC m=+811.649020337" watchObservedRunningTime="2026-09-09 12:33:11.956224455 +0000 UTC m=+811.652789610"
Sep 09 12:33:12.194390 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:12.194315    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:12.199293 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:12.199270    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:12.933683 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:12.933652    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:12.934703 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:12.934683    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:15.133594 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.133557    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"]
Sep 09 12:33:15.137969 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.137945    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:15.140015 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.139995    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-1e223f-serving-cert\""
Sep 09 12:33:15.156364 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.156336    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"]
Sep 09 12:33:15.302718 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.302681    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd4z4\" (UniqueName: \"kubernetes.io/projected/38f88501-a343-42df-870c-c6fa0bfcbf25-kube-api-access-nd4z4\") pod \"maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:15.302945 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.302727    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38f88501-a343-42df-870c-c6fa0bfcbf25-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:15.302945 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.302783    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38f88501-a343-42df-870c-c6fa0bfcbf25-maas-api-tls\") pod \"maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:15.302945 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.302800    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38f88501-a343-42df-870c-c6fa0bfcbf25-metrics-certs\") pod \"maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:15.404200 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.404117    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd4z4\" (UniqueName: \"kubernetes.io/projected/38f88501-a343-42df-870c-c6fa0bfcbf25-kube-api-access-nd4z4\") pod \"maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:15.404200 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.404166    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38f88501-a343-42df-870c-c6fa0bfcbf25-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:15.404421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.404207    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38f88501-a343-42df-870c-c6fa0bfcbf25-maas-api-tls\") pod \"maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:15.404421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.404222    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38f88501-a343-42df-870c-c6fa0bfcbf25-metrics-certs\") pod \"maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:15.404999 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.404973    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38f88501-a343-42df-870c-c6fa0bfcbf25-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:15.406891 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.406865    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38f88501-a343-42df-870c-c6fa0bfcbf25-maas-api-tls\") pod \"maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:15.407004 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.406985    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38f88501-a343-42df-870c-c6fa0bfcbf25-metrics-certs\") pod \"maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:15.413704 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.413675    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd4z4\" (UniqueName: \"kubernetes.io/projected/38f88501-a343-42df-870c-c6fa0bfcbf25-kube-api-access-nd4z4\") pod \"maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:15.449444 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.449410    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:15.739501 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.739471    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"]
Sep 09 12:33:15.741247 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:33:15.741216    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38f88501_a343_42df_870c_c6fa0bfcbf25.slice/crio-c69e27b7d72af77d4508ccd5bcdec6c2df1e8dd155437d2e40bcdd8798564a1d WatchSource:0}: Error finding container c69e27b7d72af77d4508ccd5bcdec6c2df1e8dd155437d2e40bcdd8798564a1d: Status 404 returned error can't find the container with id c69e27b7d72af77d4508ccd5bcdec6c2df1e8dd155437d2e40bcdd8798564a1d
Sep 09 12:33:15.963838 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:15.963792    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h" event={"ID":"38f88501-a343-42df-870c-c6fa0bfcbf25","Type":"ContainerStarted","Data":"c69e27b7d72af77d4508ccd5bcdec6c2df1e8dd155437d2e40bcdd8798564a1d"}
Sep 09 12:33:16.972656 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:16.972619    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h" event={"ID":"38f88501-a343-42df-870c-c6fa0bfcbf25","Type":"ContainerStarted","Data":"78ed20d59d38508b9976533223bdabdc42bef69985db00cd55a23d5679c1f812"}
Sep 09 12:33:16.973193 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:16.972798    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:16.991524 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:16.991462    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h" podStartSLOduration=1.399981827 podStartE2EDuration="1.991442019s" totalImagesPullingTime="591.460192ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:33:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:33:16.991229055 +0000 UTC m=+816.687794209" watchObservedRunningTime="2026-09-09 12:33:16.991442019 +0000 UTC m=+816.688007172"
Sep 09 12:33:19.598842 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.598799    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"]
Sep 09 12:33:19.602921 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.602900    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.605501 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.605476    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-e917b0-openshift-default-dockercfg-r8g49\""
Sep 09 12:33:19.613103 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.613077    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"]
Sep 09 12:33:19.745966 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.745926    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-workload-certs\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.746148 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.745972    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-data\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.746148 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.745999    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-podinfo\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.746148 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.746114    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-token\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.746330 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.746194    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-workload-socket\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.746330 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.746220    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-envoy\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.746330 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.746257    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istiod-ca-cert\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.746330 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.746281    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjjj7\" (UniqueName: \"kubernetes.io/projected/63c0d03c-3936-49e2-9c39-d5328cf5e99b-kube-api-access-kjjj7\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.746496 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.746341    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-credential-socket\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.847843 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.847803    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-credential-socket\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.847843 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.847885    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-workload-certs\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.848155 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.847913    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-data\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.848155 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.847945    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-podinfo\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.848155 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.848037    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-token\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.848155 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.848127    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-workload-socket\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.848155 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.848154    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-envoy\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.848421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.848190    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istiod-ca-cert\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.848421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.848216    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjjj7\" (UniqueName: \"kubernetes.io/projected/63c0d03c-3936-49e2-9c39-d5328cf5e99b-kube-api-access-kjjj7\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.848421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.848296    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-credential-socket\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.848421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.848345    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-workload-certs\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.848421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.848363    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-data\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.848675 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.848615    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-workload-socket\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.849109 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.849052    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istiod-ca-cert\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.851607 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.851560    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-podinfo\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.851775 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.851743    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-envoy\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.858919 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.858888    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjjj7\" (UniqueName: \"kubernetes.io/projected/63c0d03c-3936-49e2-9c39-d5328cf5e99b-kube-api-access-kjjj7\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.859168 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.859149    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-token\") pod \"e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") " pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:19.918169 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:19.918134    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:20.211252 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:20.211211    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"]
Sep 09 12:33:20.211922 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:33:20.211896    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63c0d03c_3936_49e2_9c39_d5328cf5e99b.slice/crio-0547ddf2a106285dfa45286c155afe52dbcc175b5ddaabc14d1a6111f73ff5b9 WatchSource:0}: Error finding container 0547ddf2a106285dfa45286c155afe52dbcc175b5ddaabc14d1a6111f73ff5b9: Status 404 returned error can't find the container with id 0547ddf2a106285dfa45286c155afe52dbcc175b5ddaabc14d1a6111f73ff5b9
Sep 09 12:33:20.214342 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:20.214300    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:33:20.214422 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:20.214396    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:33:20.214469 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:20.214441    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:33:21.000737 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:21.000696    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c" event={"ID":"63c0d03c-3936-49e2-9c39-d5328cf5e99b","Type":"ContainerStarted","Data":"f8b79e66c3be91f2e56d0a2f70992fd00265826f2b86e620aea3a70e294201a9"}
Sep 09 12:33:21.001223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:21.000760    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c" event={"ID":"63c0d03c-3936-49e2-9c39-d5328cf5e99b","Type":"ContainerStarted","Data":"0547ddf2a106285dfa45286c155afe52dbcc175b5ddaabc14d1a6111f73ff5b9"}
Sep 09 12:33:21.035991 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:21.035940    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c" podStartSLOduration=2.035924502 podStartE2EDuration="2.035924502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:33:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:33:21.034933537 +0000 UTC m=+820.731498690" watchObservedRunningTime="2026-09-09 12:33:21.035924502 +0000 UTC m=+820.732489653"
Sep 09 12:33:21.918977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:21.918932    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:21.925723 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:21.925695    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:22.009414 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:22.009347    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:22.012771 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:22.012747    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:22.988363 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:22.988318    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:33:24.022674 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.022643    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"]
Sep 09 12:33:24.027391 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.027366    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:24.030428 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.030406    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f09b7dee-serving-cert\""
Sep 09 12:33:24.042534 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.042505    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"]
Sep 09 12:33:24.104490 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.104458    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7pvc\" (UniqueName: \"kubernetes.io/projected/d121e6f5-3b69-44e2-ba2f-68cdeae53227-kube-api-access-r7pvc\") pod \"maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:24.104688 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.104506    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d121e6f5-3b69-44e2-ba2f-68cdeae53227-metrics-certs\") pod \"maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:24.104688 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.104553    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d121e6f5-3b69-44e2-ba2f-68cdeae53227-openshift-service-ca\") pod \"maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:24.104780 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.104730    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d121e6f5-3b69-44e2-ba2f-68cdeae53227-maas-api-tls\") pod \"maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:24.205550 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.205515    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7pvc\" (UniqueName: \"kubernetes.io/projected/d121e6f5-3b69-44e2-ba2f-68cdeae53227-kube-api-access-r7pvc\") pod \"maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:24.205550 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.205556    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d121e6f5-3b69-44e2-ba2f-68cdeae53227-metrics-certs\") pod \"maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:24.205816 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.205587    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d121e6f5-3b69-44e2-ba2f-68cdeae53227-openshift-service-ca\") pod \"maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:24.205816 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.205631    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d121e6f5-3b69-44e2-ba2f-68cdeae53227-maas-api-tls\") pod \"maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:24.205816 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:33:24.205760    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-serving-cert: secret "maas-api-e2e-ait-f09b7dee-serving-cert" not found
Sep 09 12:33:24.205998 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:33:24.205824    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d121e6f5-3b69-44e2-ba2f-68cdeae53227-maas-api-tls podName:d121e6f5-3b69-44e2-ba2f-68cdeae53227 nodeName:}" failed. No retries permitted until 2026-09-09 12:33:24.705805345 +0000 UTC m=+824.402370500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d121e6f5-3b69-44e2-ba2f-68cdeae53227-maas-api-tls") pod "maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9" (UID: "d121e6f5-3b69-44e2-ba2f-68cdeae53227") : secret "maas-api-e2e-ait-f09b7dee-serving-cert" not found
Sep 09 12:33:24.206499 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.206471    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d121e6f5-3b69-44e2-ba2f-68cdeae53227-openshift-service-ca\") pod \"maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:24.208481 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.208456    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d121e6f5-3b69-44e2-ba2f-68cdeae53227-metrics-certs\") pod \"maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:24.214809 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.214781    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7pvc\" (UniqueName: \"kubernetes.io/projected/d121e6f5-3b69-44e2-ba2f-68cdeae53227-kube-api-access-r7pvc\") pod \"maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:24.712061 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.711896    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d121e6f5-3b69-44e2-ba2f-68cdeae53227-maas-api-tls\") pod \"maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:24.715678 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.715632    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d121e6f5-3b69-44e2-ba2f-68cdeae53227-maas-api-tls\") pod \"maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:24.942643 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:24.942603    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:25.261727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.261687    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"]
Sep 09 12:33:25.265009 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:33:25.264955    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd121e6f5_3b69_44e2_ba2f_68cdeae53227.slice/crio-c8e3de30d6c2f69b805e6d056598bd8ce5e7d50f774068ed70ce26298b61f0e2 WatchSource:0}: Error finding container c8e3de30d6c2f69b805e6d056598bd8ce5e7d50f774068ed70ce26298b61f0e2: Status 404 returned error can't find the container with id c8e3de30d6c2f69b805e6d056598bd8ce5e7d50f774068ed70ce26298b61f0e2
Sep 09 12:33:25.265925 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.265893    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"]
Sep 09 12:33:25.272883 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.271944    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:25.278541 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.274882    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-e917b0-serving-cert\""
Sep 09 12:33:25.278541 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.278434    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"]
Sep 09 12:33:25.421550 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.421513    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15521e6b-3cc6-44a8-9e54-e8bda208836e-metrics-certs\") pod \"maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:25.421733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.421565    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15521e6b-3cc6-44a8-9e54-e8bda208836e-openshift-service-ca\") pod \"maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:25.421733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.421627    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15521e6b-3cc6-44a8-9e54-e8bda208836e-maas-api-tls\") pod \"maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:25.421814 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.421729    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjdvd\" (UniqueName: \"kubernetes.io/projected/15521e6b-3cc6-44a8-9e54-e8bda208836e-kube-api-access-fjdvd\") pod \"maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:25.522443 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.522402    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjdvd\" (UniqueName: \"kubernetes.io/projected/15521e6b-3cc6-44a8-9e54-e8bda208836e-kube-api-access-fjdvd\") pod \"maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:25.522621 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.522495    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15521e6b-3cc6-44a8-9e54-e8bda208836e-metrics-certs\") pod \"maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:25.522621 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.522526    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15521e6b-3cc6-44a8-9e54-e8bda208836e-openshift-service-ca\") pod \"maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:25.522621 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.522569    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15521e6b-3cc6-44a8-9e54-e8bda208836e-maas-api-tls\") pod \"maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:25.523225 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.523205    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15521e6b-3cc6-44a8-9e54-e8bda208836e-openshift-service-ca\") pod \"maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:25.525270 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.525249    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15521e6b-3cc6-44a8-9e54-e8bda208836e-metrics-certs\") pod \"maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:25.525344 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.525327    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15521e6b-3cc6-44a8-9e54-e8bda208836e-maas-api-tls\") pod \"maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:25.531101 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.531038    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjdvd\" (UniqueName: \"kubernetes.io/projected/15521e6b-3cc6-44a8-9e54-e8bda208836e-kube-api-access-fjdvd\") pod \"maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:25.616244 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.616208    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:25.915071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:25.915042    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"]
Sep 09 12:33:25.916062 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:33:25.916031    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15521e6b_3cc6_44a8_9e54_e8bda208836e.slice/crio-8185dabf953b82f67b34fda3aca7a2aa6c548a4155265db879ca7df6d68eed33 WatchSource:0}: Error finding container 8185dabf953b82f67b34fda3aca7a2aa6c548a4155265db879ca7df6d68eed33: Status 404 returned error can't find the container with id 8185dabf953b82f67b34fda3aca7a2aa6c548a4155265db879ca7df6d68eed33
Sep 09 12:33:26.039381 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:26.039338    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h" event={"ID":"15521e6b-3cc6-44a8-9e54-e8bda208836e","Type":"ContainerStarted","Data":"8185dabf953b82f67b34fda3aca7a2aa6c548a4155265db879ca7df6d68eed33"}
Sep 09 12:33:26.041947 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:26.041733    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9" event={"ID":"d121e6f5-3b69-44e2-ba2f-68cdeae53227","Type":"ContainerStarted","Data":"560dcdc9fde0d74300fe35a8ce7b1192b38aaa3b2b73a0fd6df1daf14e8543fc"}
Sep 09 12:33:26.041947 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:26.041813    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9" event={"ID":"d121e6f5-3b69-44e2-ba2f-68cdeae53227","Type":"ContainerStarted","Data":"c8e3de30d6c2f69b805e6d056598bd8ce5e7d50f774068ed70ce26298b61f0e2"}
Sep 09 12:33:26.063049 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:26.061410    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9" podStartSLOduration=2.578308024 podStartE2EDuration="3.061374488s" totalImagesPullingTime="483.066464ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:33:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:33:26.058000238 +0000 UTC m=+825.754565395" watchObservedRunningTime="2026-09-09 12:33:26.061374488 +0000 UTC m=+825.757939643"
Sep 09 12:33:27.049362 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:27.049321    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h" event={"ID":"15521e6b-3cc6-44a8-9e54-e8bda208836e","Type":"ContainerStarted","Data":"bdb9d7d3fac454b7f1f5e2fe88a6cda251749151ab24f0b43156e40379b6ecf1"}
Sep 09 12:33:27.049781 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:27.049490    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:27.049781 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:27.049716    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:27.067496 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:27.067433    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h" podStartSLOduration=1.66052325 podStartE2EDuration="2.067414684s" totalImagesPullingTime="406.891434ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:33:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:33:27.065636138 +0000 UTC m=+826.762201289" watchObservedRunningTime="2026-09-09 12:33:27.067414684 +0000 UTC m=+826.763979836"
Sep 09 12:33:28.976020 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:28.975977    2582 patch_prober.go:28] interesting pod/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 12:33:28.976413 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:28.976044    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn" podUID="afcba06e-529f-471a-a7fb-9df792f78662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 12:33:31.074946 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:31.074913    2582 generic.go:356] "Generic (PLEG): container finished" podID="afcba06e-529f-471a-a7fb-9df792f78662" containerID="bd44f442fadac3da25165cf740311b94cbeb35336826b153fca717cbac030d64" exitCode=0
Sep 09 12:33:31.075523 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:31.074999    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn" event={"ID":"afcba06e-529f-471a-a7fb-9df792f78662","Type":"ContainerDied","Data":"bd44f442fadac3da25165cf740311b94cbeb35336826b153fca717cbac030d64"}
Sep 09 12:33:31.076136 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:31.076119    2582 scope.go:117] "RemoveContainer" containerID="bd44f442fadac3da25165cf740311b94cbeb35336826b153fca717cbac030d64"
Sep 09 12:33:31.080352 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:31.077629    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:33:31.080352 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:31.077722    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:33:31.080352 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:31.077769    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:33:32.088238 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:32.088187    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn" event={"ID":"afcba06e-529f-471a-a7fb-9df792f78662","Type":"ContainerStarted","Data":"150f932af157012650071abede143c5a2b361367cd993ba301f161e02acb8e17"}
Sep 09 12:33:32.194577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:32.194535    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:32.194818 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:32.194792    2582 patch_prober.go:28] interesting pod/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 09 12:33:32.194926 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:32.194868    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn" podUID="afcba06e-529f-471a-a7fb-9df792f78662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 09 12:33:33.061619 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:33.061587    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:33:33.062229 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:33.062208    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:33:33.195359 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:33.195320    2582 patch_prober.go:28] interesting pod/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 09 12:33:33.195966 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:33.195932    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn" podUID="afcba06e-529f-471a-a7fb-9df792f78662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 09 12:33:34.194865 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:34.194810    2582 patch_prober.go:28] interesting pod/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 09 12:33:34.195087 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:34.194901    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn" podUID="afcba06e-529f-471a-a7fb-9df792f78662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 09 12:33:35.195001 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:35.194952    2582 patch_prober.go:28] interesting pod/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 09 12:33:35.195536 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:35.195025    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn" podUID="afcba06e-529f-471a-a7fb-9df792f78662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 09 12:33:36.195951 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:36.195908    2582 patch_prober.go:28] interesting pod/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 09 12:33:36.196563 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:36.196531    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn" podUID="afcba06e-529f-471a-a7fb-9df792f78662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 09 12:33:37.194915 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:37.194873    2582 patch_prober.go:28] interesting pod/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 09 12:33:37.195122 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:37.194938    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn" podUID="afcba06e-529f-471a-a7fb-9df792f78662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 09 12:33:38.051080 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:38.051034    2582 patch_prober.go:28] interesting pod/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 12:33:38.051572 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:38.051113    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c" podUID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 12:33:38.200321 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:38.200287    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:38.200673 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:38.200639    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:38.201503 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:38.201484    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:33:39.412779 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.412748    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"]
Sep 09 12:33:39.416574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.416553    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.419971 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.419746    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-5e5923-openshift-default-dockercfg-mhmzs\""
Sep 09 12:33:39.432391 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.432365    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"]
Sep 09 12:33:39.476436 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.476393    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-token\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.476609 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.476465    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-envoy\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.476609 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.476527    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-workload-socket\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.476705 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.476611    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9f299b8-c998-444d-b2f1-3b47e15b4550-istiod-ca-cert\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.476705 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.476642    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-workload-certs\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.476705 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.476679    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-credential-socket\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.476825 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.476791    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8sck\" (UniqueName: \"kubernetes.io/projected/f9f299b8-c998-444d-b2f1-3b47e15b4550-kube-api-access-k8sck\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.476960 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.476936    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-podinfo\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.477034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.476978    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-data\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.578426 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.578384    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-podinfo\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.578617 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.578437    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-data\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.578617 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.578471    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-token\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.578617 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.578503    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-envoy\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.578617 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.578556    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-workload-socket\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.578617 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.578607    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9f299b8-c998-444d-b2f1-3b47e15b4550-istiod-ca-cert\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.578906 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.578630    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-workload-certs\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.578906 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.578664    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-credential-socket\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.578906 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.578706    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8sck\" (UniqueName: \"kubernetes.io/projected/f9f299b8-c998-444d-b2f1-3b47e15b4550-kube-api-access-k8sck\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.579069 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.578936    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-data\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.579232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.579196    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-workload-socket\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.579232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.579221    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-workload-certs\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.579353 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.579333    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-credential-socket\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.579752 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.579680    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9f299b8-c998-444d-b2f1-3b47e15b4550-istiod-ca-cert\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.581507 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.581250    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-podinfo\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.581929 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.581700    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-envoy\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.588520 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.588485    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8sck\" (UniqueName: \"kubernetes.io/projected/f9f299b8-c998-444d-b2f1-3b47e15b4550-kube-api-access-k8sck\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.588709 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.588634    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-token\") pod \"e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") " pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:39.732980 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:39.732844    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:40.024712 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:40.023735    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"]
Sep 09 12:33:40.025339 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:33:40.025311    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f299b8_c998_444d_b2f1_3b47e15b4550.slice/crio-71895cfc86f81ab8f22d1af245b53064e1cf52648cc0b595acf28df3a1596112 WatchSource:0}: Error finding container 71895cfc86f81ab8f22d1af245b53064e1cf52648cc0b595acf28df3a1596112: Status 404 returned error can't find the container with id 71895cfc86f81ab8f22d1af245b53064e1cf52648cc0b595acf28df3a1596112
Sep 09 12:33:40.027843 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:40.027799    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:33:40.027937 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:40.027916    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:33:40.027983 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:40.027968    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:33:40.161357 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:40.161314    2582 generic.go:356] "Generic (PLEG): container finished" podID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" containerID="f8b79e66c3be91f2e56d0a2f70992fd00265826f2b86e620aea3a70e294201a9" exitCode=0
Sep 09 12:33:40.161551 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:40.161402    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c" event={"ID":"63c0d03c-3936-49e2-9c39-d5328cf5e99b","Type":"ContainerDied","Data":"f8b79e66c3be91f2e56d0a2f70992fd00265826f2b86e620aea3a70e294201a9"}
Sep 09 12:33:40.161981 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:40.161955    2582 scope.go:117] "RemoveContainer" containerID="f8b79e66c3be91f2e56d0a2f70992fd00265826f2b86e620aea3a70e294201a9"
Sep 09 12:33:40.163231 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:40.163198    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:33:40.163332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:40.163281    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:33:40.163380 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:40.163354    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:33:40.163912 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:40.163840    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb" event={"ID":"f9f299b8-c998-444d-b2f1-3b47e15b4550","Type":"ContainerStarted","Data":"f820dfe10066e4e2a2f6ad261fa24fa62c12a408212c43fe125294966705ca17"}
Sep 09 12:33:40.164004 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:40.163941    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb" event={"ID":"f9f299b8-c998-444d-b2f1-3b47e15b4550","Type":"ContainerStarted","Data":"71895cfc86f81ab8f22d1af245b53064e1cf52648cc0b595acf28df3a1596112"}
Sep 09 12:33:40.733629 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:40.733596    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:41.173242 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:41.173203    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c" event={"ID":"63c0d03c-3936-49e2-9c39-d5328cf5e99b","Type":"ContainerStarted","Data":"1e3d77f449a70d2cd7bda90e2a6d72737aeb5742a41b7786058044076e48af64"}
Sep 09 12:33:41.194903 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:41.194815    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb" podStartSLOduration=2.194790862 podStartE2EDuration="2.194790862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:33:40.200253314 +0000 UTC m=+839.896818493" watchObservedRunningTime="2026-09-09 12:33:41.194790862 +0000 UTC m=+840.891356014"
Sep 09 12:33:41.739366 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:41.739334    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:41.919332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:41.919298    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:41.919494 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:41.919476    2582 patch_prober.go:28] interesting pod/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 09 12:33:41.919546 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:41.919519    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c" podUID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 09 12:33:42.183819 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:42.183651    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:42.188583 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:42.188558    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:33:42.919149 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:42.919116    2582 patch_prober.go:28] interesting pod/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 09 12:33:42.919557 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:42.919169    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c" podUID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 09 12:33:43.918521 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:43.918487    2582 patch_prober.go:28] interesting pod/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 09 12:33:43.918693 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:43.918545    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c" podUID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 09 12:33:44.919063 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:44.919030    2582 patch_prober.go:28] interesting pod/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 09 12:33:44.919453 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:44.919084    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c" podUID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 09 12:33:45.918555 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:45.918505    2582 patch_prober.go:28] interesting pod/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 09 12:33:45.918713 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:45.918571    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c" podUID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 09 12:33:46.923810 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:46.923774    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:46.924274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:46.924109    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:33:46.925017 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:33:46.924997    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:34:04.732825 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:04.732789    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"]
Sep 09 12:34:04.733351 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:04.733073    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9" podUID="d121e6f5-3b69-44e2-ba2f-68cdeae53227" containerName="maas-api" containerID="cri-o://560dcdc9fde0d74300fe35a8ce7b1192b38aaa3b2b73a0fd6df1daf14e8543fc" gracePeriod=30
Sep 09 12:34:05.000527 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.000502    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:34:05.165969 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.165930    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d121e6f5-3b69-44e2-ba2f-68cdeae53227-openshift-service-ca\") pod \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") "
Sep 09 12:34:05.166136 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.165989    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d121e6f5-3b69-44e2-ba2f-68cdeae53227-metrics-certs\") pod \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") "
Sep 09 12:34:05.166136 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.166061    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7pvc\" (UniqueName: \"kubernetes.io/projected/d121e6f5-3b69-44e2-ba2f-68cdeae53227-kube-api-access-r7pvc\") pod \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") "
Sep 09 12:34:05.166136 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.166094    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d121e6f5-3b69-44e2-ba2f-68cdeae53227-maas-api-tls\") pod \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\" (UID: \"d121e6f5-3b69-44e2-ba2f-68cdeae53227\") "
Sep 09 12:34:05.166306 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.166237    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d121e6f5-3b69-44e2-ba2f-68cdeae53227-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d121e6f5-3b69-44e2-ba2f-68cdeae53227" (UID: "d121e6f5-3b69-44e2-ba2f-68cdeae53227"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:34:05.166369 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.166351    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d121e6f5-3b69-44e2-ba2f-68cdeae53227-openshift-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:34:05.168474 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.168439    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d121e6f5-3b69-44e2-ba2f-68cdeae53227-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d121e6f5-3b69-44e2-ba2f-68cdeae53227" (UID: "d121e6f5-3b69-44e2-ba2f-68cdeae53227"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:34:05.168604 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.168519    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d121e6f5-3b69-44e2-ba2f-68cdeae53227-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d121e6f5-3b69-44e2-ba2f-68cdeae53227" (UID: "d121e6f5-3b69-44e2-ba2f-68cdeae53227"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:34:05.168604 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.168524    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d121e6f5-3b69-44e2-ba2f-68cdeae53227-kube-api-access-r7pvc" (OuterVolumeSpecName: "kube-api-access-r7pvc") pod "d121e6f5-3b69-44e2-ba2f-68cdeae53227" (UID: "d121e6f5-3b69-44e2-ba2f-68cdeae53227"). InnerVolumeSpecName "kube-api-access-r7pvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:34:05.267676 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.267645    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7pvc\" (UniqueName: \"kubernetes.io/projected/d121e6f5-3b69-44e2-ba2f-68cdeae53227-kube-api-access-r7pvc\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:34:05.267676 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.267673    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d121e6f5-3b69-44e2-ba2f-68cdeae53227-maas-api-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:34:05.267676 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.267682    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d121e6f5-3b69-44e2-ba2f-68cdeae53227-metrics-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:34:05.351138 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.351110    2582 generic.go:356] "Generic (PLEG): container finished" podID="d121e6f5-3b69-44e2-ba2f-68cdeae53227" containerID="560dcdc9fde0d74300fe35a8ce7b1192b38aaa3b2b73a0fd6df1daf14e8543fc" exitCode=0
Sep 09 12:34:05.351289 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.351161    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"
Sep 09 12:34:05.351289 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.351183    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9" event={"ID":"d121e6f5-3b69-44e2-ba2f-68cdeae53227","Type":"ContainerDied","Data":"560dcdc9fde0d74300fe35a8ce7b1192b38aaa3b2b73a0fd6df1daf14e8543fc"}
Sep 09 12:34:05.351289 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.351250    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9" event={"ID":"d121e6f5-3b69-44e2-ba2f-68cdeae53227","Type":"ContainerDied","Data":"c8e3de30d6c2f69b805e6d056598bd8ce5e7d50f774068ed70ce26298b61f0e2"}
Sep 09 12:34:05.351289 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.351271    2582 scope.go:117] "RemoveContainer" containerID="560dcdc9fde0d74300fe35a8ce7b1192b38aaa3b2b73a0fd6df1daf14e8543fc"
Sep 09 12:34:05.360946 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.360889    2582 scope.go:117] "RemoveContainer" containerID="560dcdc9fde0d74300fe35a8ce7b1192b38aaa3b2b73a0fd6df1daf14e8543fc"
Sep 09 12:34:05.361173 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:34:05.361157    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"560dcdc9fde0d74300fe35a8ce7b1192b38aaa3b2b73a0fd6df1daf14e8543fc\": container with ID starting with 560dcdc9fde0d74300fe35a8ce7b1192b38aaa3b2b73a0fd6df1daf14e8543fc not found: ID does not exist" containerID="560dcdc9fde0d74300fe35a8ce7b1192b38aaa3b2b73a0fd6df1daf14e8543fc"
Sep 09 12:34:05.361219 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.361181    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"560dcdc9fde0d74300fe35a8ce7b1192b38aaa3b2b73a0fd6df1daf14e8543fc"} err="failed to get container status \"560dcdc9fde0d74300fe35a8ce7b1192b38aaa3b2b73a0fd6df1daf14e8543fc\": rpc error: code = NotFound desc = could not find container \"560dcdc9fde0d74300fe35a8ce7b1192b38aaa3b2b73a0fd6df1daf14e8543fc\": container with ID starting with 560dcdc9fde0d74300fe35a8ce7b1192b38aaa3b2b73a0fd6df1daf14e8543fc not found: ID does not exist"
Sep 09 12:34:05.374727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.374700    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"]
Sep 09 12:34:05.379575 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:05.379555    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f09b7dee-66f78594f6-6wzr9"]
Sep 09 12:34:06.078738 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.078647    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"]
Sep 09 12:34:06.079366 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.079345    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d121e6f5-3b69-44e2-ba2f-68cdeae53227" containerName="maas-api"
Sep 09 12:34:06.079439 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.079381    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d121e6f5-3b69-44e2-ba2f-68cdeae53227" containerName="maas-api"
Sep 09 12:34:06.079514 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.079503    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d121e6f5-3b69-44e2-ba2f-68cdeae53227" containerName="maas-api"
Sep 09 12:34:06.083351 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.083325    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.085536 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.085514    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-dbdca9a8-gw-openshift-default-dockercfg-57tf4\""
Sep 09 12:34:06.094756 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.094725    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"]
Sep 09 12:34:06.176819 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.176786    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-credential-socket\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.176819 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.176822    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv66r\" (UniqueName: \"kubernetes.io/projected/09b7894b-73cb-499d-92ad-d72b0e501e48-kube-api-access-qv66r\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.177071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.176894    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-token\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.177071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.176964    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-workload-socket\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.177071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.176993    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-podinfo\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.177071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.177007    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-data\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.177071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.177032    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-envoy\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.177330 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.177078    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-workload-certs\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.177330 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.177105    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b7894b-73cb-499d-92ad-d72b0e501e48-istiod-ca-cert\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.278294 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.278251    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-credential-socket\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.278294 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.278296    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv66r\" (UniqueName: \"kubernetes.io/projected/09b7894b-73cb-499d-92ad-d72b0e501e48-kube-api-access-qv66r\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.278650 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.278349    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-token\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.278650 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.278517    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-workload-socket\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.278650 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.278559    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-podinfo\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.278650 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.278583    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-data\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.278650 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.278615    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-envoy\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.278965 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.278654    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-workload-certs\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.278965 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.278683    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b7894b-73cb-499d-92ad-d72b0e501e48-istiod-ca-cert\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.278965 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.278716    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-credential-socket\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.278965 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.278954    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-workload-socket\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.279167 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.279072    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-data\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.279224 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.279188    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-workload-certs\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.279503 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.279400    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b7894b-73cb-499d-92ad-d72b0e501e48-istiod-ca-cert\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.281150 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.281125    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-envoy\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.281269 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.281251    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-podinfo\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.286355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.286328    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-token\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.286447 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.286360    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv66r\" (UniqueName: \"kubernetes.io/projected/09b7894b-73cb-499d-92ad-d72b0e501e48-kube-api-access-qv66r\") pod \"e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") " pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.359425 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.359363    2582 generic.go:356] "Generic (PLEG): container finished" podID="f9f299b8-c998-444d-b2f1-3b47e15b4550" containerID="f820dfe10066e4e2a2f6ad261fa24fa62c12a408212c43fe125294966705ca17" exitCode=0
Sep 09 12:34:06.359425 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.359416    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb" event={"ID":"f9f299b8-c998-444d-b2f1-3b47e15b4550","Type":"ContainerDied","Data":"f820dfe10066e4e2a2f6ad261fa24fa62c12a408212c43fe125294966705ca17"}
Sep 09 12:34:06.359880 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.359866    2582 scope.go:117] "RemoveContainer" containerID="f820dfe10066e4e2a2f6ad261fa24fa62c12a408212c43fe125294966705ca17"
Sep 09 12:34:06.361008 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.360981    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:34:06.361068 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.361047    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:34:06.361107 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.361079    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:34:06.398622 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.398601    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:06.685691 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.685047    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"]
Sep 09 12:34:06.685896 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:34:06.685840    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09b7894b_73cb_499d_92ad_d72b0e501e48.slice/crio-adba5d912baa56e792e90eed7b132aef9d66f9db66160442dbbf61a59ce6ce53 WatchSource:0}: Error finding container adba5d912baa56e792e90eed7b132aef9d66f9db66160442dbbf61a59ce6ce53: Status 404 returned error can't find the container with id adba5d912baa56e792e90eed7b132aef9d66f9db66160442dbbf61a59ce6ce53
Sep 09 12:34:06.688689 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.688649    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:34:06.688796 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.688737    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:34:06.688796 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.688783    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:34:06.828452 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:06.828414    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d121e6f5-3b69-44e2-ba2f-68cdeae53227" path="/var/lib/kubelet/pods/d121e6f5-3b69-44e2-ba2f-68cdeae53227/volumes"
Sep 09 12:34:07.367307 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:07.367274    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d" event={"ID":"09b7894b-73cb-499d-92ad-d72b0e501e48","Type":"ContainerStarted","Data":"b7376fd07fd0dd15212c6c5ed49ce697fb756315c0398fe6ff669d9094393a16"}
Sep 09 12:34:07.367733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:07.367325    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d" event={"ID":"09b7894b-73cb-499d-92ad-d72b0e501e48","Type":"ContainerStarted","Data":"adba5d912baa56e792e90eed7b132aef9d66f9db66160442dbbf61a59ce6ce53"}
Sep 09 12:34:07.369516 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:07.369484    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb" event={"ID":"f9f299b8-c998-444d-b2f1-3b47e15b4550","Type":"ContainerStarted","Data":"08b755407a35bed1245530d7b63a09ef2c7af18db3be8b50f18a7f41fea7fbe5"}
Sep 09 12:34:07.399674 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:07.399646    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:07.403897 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:07.403836    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d" podStartSLOduration=1.403817603 podStartE2EDuration="1.403817603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:34:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:34:07.400101855 +0000 UTC m=+867.096667009" watchObservedRunningTime="2026-09-09 12:34:07.403817603 +0000 UTC m=+867.100382779"
Sep 09 12:34:07.405304 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:07.405285    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:07.733803 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:07.733719    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:34:07.734640 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:07.734619    2582 patch_prober.go:28] interesting pod/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 09 12:34:07.734741 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:07.734671    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb" podUID="f9f299b8-c998-444d-b2f1-3b47e15b4550" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 09 12:34:08.387836 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:08.387791    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:08.394203 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:08.394162    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:08.733421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:08.733338    2582 patch_prober.go:28] interesting pod/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 09 12:34:08.733421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:08.733390    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb" podUID="f9f299b8-c998-444d-b2f1-3b47e15b4550" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 09 12:34:09.734147 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:09.734114    2582 patch_prober.go:28] interesting pod/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 09 12:34:09.734526 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:09.734166    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb" podUID="f9f299b8-c998-444d-b2f1-3b47e15b4550" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 09 12:34:10.733508 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:10.733463    2582 patch_prober.go:28] interesting pod/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 09 12:34:10.733691 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:10.733525    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb" podUID="f9f299b8-c998-444d-b2f1-3b47e15b4550" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 09 12:34:11.733821 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:11.733779    2582 patch_prober.go:28] interesting pod/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 09 12:34:11.734304 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:11.733844    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb" podUID="f9f299b8-c998-444d-b2f1-3b47e15b4550" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 09 12:34:12.184486 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:12.184445    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:34:12.734070 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:12.734038    2582 patch_prober.go:28] interesting pod/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 09 12:34:12.734435 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:12.734095    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb" podUID="f9f299b8-c998-444d-b2f1-3b47e15b4550" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 09 12:34:13.737908 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:13.737877    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:34:13.738901 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:13.738882    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:34:27.516304 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:27.516272    2582 generic.go:356] "Generic (PLEG): container finished" podID="09b7894b-73cb-499d-92ad-d72b0e501e48" containerID="b7376fd07fd0dd15212c6c5ed49ce697fb756315c0398fe6ff669d9094393a16" exitCode=0
Sep 09 12:34:27.516718 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:27.516345    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d" event={"ID":"09b7894b-73cb-499d-92ad-d72b0e501e48","Type":"ContainerDied","Data":"b7376fd07fd0dd15212c6c5ed49ce697fb756315c0398fe6ff669d9094393a16"}
Sep 09 12:34:27.516989 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:27.516740    2582 scope.go:117] "RemoveContainer" containerID="b7376fd07fd0dd15212c6c5ed49ce697fb756315c0398fe6ff669d9094393a16"
Sep 09 12:34:27.517834 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:27.517808    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:34:27.517955 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:27.517905    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:34:27.517955 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:27.517940    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:34:28.524601 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:28.524565    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d" event={"ID":"09b7894b-73cb-499d-92ad-d72b0e501e48","Type":"ContainerStarted","Data":"38cfc10a875fae6763e5d666c7958bf193e2daa9dd53ba4bd95c296cdc3753ae"}
Sep 09 12:34:29.399330 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:29.399283    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:29.399604 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:29.399577    2582 patch_prober.go:28] interesting pod/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 09 12:34:29.399686 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:29.399639    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d" podUID="09b7894b-73cb-499d-92ad-d72b0e501e48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 09 12:34:30.399919 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:30.399879    2582 patch_prober.go:28] interesting pod/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 09 12:34:30.400543 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:30.400510    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d" podUID="09b7894b-73cb-499d-92ad-d72b0e501e48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 09 12:34:31.400056 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:31.400011    2582 patch_prober.go:28] interesting pod/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 09 12:34:31.400540 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:31.400083    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d" podUID="09b7894b-73cb-499d-92ad-d72b0e501e48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 09 12:34:32.399409 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:32.399325    2582 patch_prober.go:28] interesting pod/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 09 12:34:32.399677 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:32.399645    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d" podUID="09b7894b-73cb-499d-92ad-d72b0e501e48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 09 12:34:33.399259 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:33.399213    2582 patch_prober.go:28] interesting pod/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 09 12:34:33.399764 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:33.399306    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d" podUID="09b7894b-73cb-499d-92ad-d72b0e501e48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 09 12:34:34.403184 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:34.403154    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:34.403731 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:34.403544    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:34.404355 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:34.404337    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:40.268826 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:40.268792    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"]
Sep 09 12:34:40.269310 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:40.269080    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d" podUID="09b7894b-73cb-499d-92ad-d72b0e501e48" containerName="istio-proxy" containerID="cri-o://38cfc10a875fae6763e5d666c7958bf193e2daa9dd53ba4bd95c296cdc3753ae" gracePeriod=30
Sep 09 12:34:45.524270 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.524243    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:45.598172 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598103    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-workload-socket\") pod \"09b7894b-73cb-499d-92ad-d72b0e501e48\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") "
Sep 09 12:34:45.598172 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598137    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-data\") pod \"09b7894b-73cb-499d-92ad-d72b0e501e48\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") "
Sep 09 12:34:45.598367 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598178    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-envoy\") pod \"09b7894b-73cb-499d-92ad-d72b0e501e48\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") "
Sep 09 12:34:45.598367 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598206    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qv66r\" (UniqueName: \"kubernetes.io/projected/09b7894b-73cb-499d-92ad-d72b0e501e48-kube-api-access-qv66r\") pod \"09b7894b-73cb-499d-92ad-d72b0e501e48\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") "
Sep 09 12:34:45.598367 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598262    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-workload-certs\") pod \"09b7894b-73cb-499d-92ad-d72b0e501e48\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") "
Sep 09 12:34:45.598367 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598286    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-token\") pod \"09b7894b-73cb-499d-92ad-d72b0e501e48\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") "
Sep 09 12:34:45.598367 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598309    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b7894b-73cb-499d-92ad-d72b0e501e48-istiod-ca-cert\") pod \"09b7894b-73cb-499d-92ad-d72b0e501e48\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") "
Sep 09 12:34:45.598612 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598377    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-podinfo\") pod \"09b7894b-73cb-499d-92ad-d72b0e501e48\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") "
Sep 09 12:34:45.598612 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598383    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "09b7894b-73cb-499d-92ad-d72b0e501e48" (UID: "09b7894b-73cb-499d-92ad-d72b0e501e48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:34:45.598612 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598428    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-credential-socket\") pod \"09b7894b-73cb-499d-92ad-d72b0e501e48\" (UID: \"09b7894b-73cb-499d-92ad-d72b0e501e48\") "
Sep 09 12:34:45.598612 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598449    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-data" (OuterVolumeSpecName: "istio-data") pod "09b7894b-73cb-499d-92ad-d72b0e501e48" (UID: "09b7894b-73cb-499d-92ad-d72b0e501e48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:34:45.598867 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598677    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "09b7894b-73cb-499d-92ad-d72b0e501e48" (UID: "09b7894b-73cb-499d-92ad-d72b0e501e48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:34:45.598867 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598690    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09b7894b-73cb-499d-92ad-d72b0e501e48-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "09b7894b-73cb-499d-92ad-d72b0e501e48" (UID: "09b7894b-73cb-499d-92ad-d72b0e501e48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:34:45.598867 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598789    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-workload-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:34:45.599204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598891    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "09b7894b-73cb-499d-92ad-d72b0e501e48" (UID: "09b7894b-73cb-499d-92ad-d72b0e501e48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:34:45.599204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598907    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b7894b-73cb-499d-92ad-d72b0e501e48-istiod-ca-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:34:45.599204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598924    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-workload-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:34:45.599204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.598942    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:34:45.600653 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.600618    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "09b7894b-73cb-499d-92ad-d72b0e501e48" (UID: "09b7894b-73cb-499d-92ad-d72b0e501e48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:34:45.600755 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.600670    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-token" (OuterVolumeSpecName: "istio-token") pod "09b7894b-73cb-499d-92ad-d72b0e501e48" (UID: "09b7894b-73cb-499d-92ad-d72b0e501e48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:34:45.600755 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.600699    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "09b7894b-73cb-499d-92ad-d72b0e501e48" (UID: "09b7894b-73cb-499d-92ad-d72b0e501e48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:34:45.600755 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.600743    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09b7894b-73cb-499d-92ad-d72b0e501e48-kube-api-access-qv66r" (OuterVolumeSpecName: "kube-api-access-qv66r") pod "09b7894b-73cb-499d-92ad-d72b0e501e48" (UID: "09b7894b-73cb-499d-92ad-d72b0e501e48"). InnerVolumeSpecName "kube-api-access-qv66r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:34:45.634414 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.634387    2582 generic.go:356] "Generic (PLEG): container finished" podID="09b7894b-73cb-499d-92ad-d72b0e501e48" containerID="38cfc10a875fae6763e5d666c7958bf193e2daa9dd53ba4bd95c296cdc3753ae" exitCode=0
Sep 09 12:34:45.634525 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.634434    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"
Sep 09 12:34:45.634525 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.634463    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d" event={"ID":"09b7894b-73cb-499d-92ad-d72b0e501e48","Type":"ContainerDied","Data":"38cfc10a875fae6763e5d666c7958bf193e2daa9dd53ba4bd95c296cdc3753ae"}
Sep 09 12:34:45.634639 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.634523    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d" event={"ID":"09b7894b-73cb-499d-92ad-d72b0e501e48","Type":"ContainerDied","Data":"adba5d912baa56e792e90eed7b132aef9d66f9db66160442dbbf61a59ce6ce53"}
Sep 09 12:34:45.634639 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.634541    2582 scope.go:117] "RemoveContainer" containerID="38cfc10a875fae6763e5d666c7958bf193e2daa9dd53ba4bd95c296cdc3753ae"
Sep 09 12:34:45.644936 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.644919    2582 scope.go:117] "RemoveContainer" containerID="b7376fd07fd0dd15212c6c5ed49ce697fb756315c0398fe6ff669d9094393a16"
Sep 09 12:34:45.655706 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.655688    2582 scope.go:117] "RemoveContainer" containerID="38cfc10a875fae6763e5d666c7958bf193e2daa9dd53ba4bd95c296cdc3753ae"
Sep 09 12:34:45.656023 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:34:45.655995    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38cfc10a875fae6763e5d666c7958bf193e2daa9dd53ba4bd95c296cdc3753ae\": container with ID starting with 38cfc10a875fae6763e5d666c7958bf193e2daa9dd53ba4bd95c296cdc3753ae not found: ID does not exist" containerID="38cfc10a875fae6763e5d666c7958bf193e2daa9dd53ba4bd95c296cdc3753ae"
Sep 09 12:34:45.656098 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.656033    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38cfc10a875fae6763e5d666c7958bf193e2daa9dd53ba4bd95c296cdc3753ae"} err="failed to get container status \"38cfc10a875fae6763e5d666c7958bf193e2daa9dd53ba4bd95c296cdc3753ae\": rpc error: code = NotFound desc = could not find container \"38cfc10a875fae6763e5d666c7958bf193e2daa9dd53ba4bd95c296cdc3753ae\": container with ID starting with 38cfc10a875fae6763e5d666c7958bf193e2daa9dd53ba4bd95c296cdc3753ae not found: ID does not exist"
Sep 09 12:34:45.656098 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.656054    2582 scope.go:117] "RemoveContainer" containerID="b7376fd07fd0dd15212c6c5ed49ce697fb756315c0398fe6ff669d9094393a16"
Sep 09 12:34:45.656339 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:34:45.656320    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7376fd07fd0dd15212c6c5ed49ce697fb756315c0398fe6ff669d9094393a16\": container with ID starting with b7376fd07fd0dd15212c6c5ed49ce697fb756315c0398fe6ff669d9094393a16 not found: ID does not exist" containerID="b7376fd07fd0dd15212c6c5ed49ce697fb756315c0398fe6ff669d9094393a16"
Sep 09 12:34:45.656387 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.656347    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7376fd07fd0dd15212c6c5ed49ce697fb756315c0398fe6ff669d9094393a16"} err="failed to get container status \"b7376fd07fd0dd15212c6c5ed49ce697fb756315c0398fe6ff669d9094393a16\": rpc error: code = NotFound desc = could not find container \"b7376fd07fd0dd15212c6c5ed49ce697fb756315c0398fe6ff669d9094393a16\": container with ID starting with b7376fd07fd0dd15212c6c5ed49ce697fb756315c0398fe6ff669d9094393a16 not found: ID does not exist"
Sep 09 12:34:45.660147 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.660127    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"]
Sep 09 12:34:45.667320 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.667295    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-dbdca9a8-gw-openshift-default-f9d5b996-z6d6d"]
Sep 09 12:34:45.699845 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.699821    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-envoy\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:34:45.699845 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.699844    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv66r\" (UniqueName: \"kubernetes.io/projected/09b7894b-73cb-499d-92ad-d72b0e501e48-kube-api-access-qv66r\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:34:45.699996 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.699868    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:34:45.699996 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.699877    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b7894b-73cb-499d-92ad-d72b0e501e48-istio-podinfo\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:34:45.699996 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:45.699885    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b7894b-73cb-499d-92ad-d72b0e501e48-credential-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:34:46.826880 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:34:46.826828    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09b7894b-73cb-499d-92ad-d72b0e501e48" path="/var/lib/kubelet/pods/09b7894b-73cb-499d-92ad-d72b0e501e48/volumes"
Sep 09 12:35:11.736954 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.736916    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"]
Sep 09 12:35:11.737633 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.737613    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09b7894b-73cb-499d-92ad-d72b0e501e48" containerName="istio-proxy"
Sep 09 12:35:11.737682 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.737636    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="09b7894b-73cb-499d-92ad-d72b0e501e48" containerName="istio-proxy"
Sep 09 12:35:11.737682 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.737648    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09b7894b-73cb-499d-92ad-d72b0e501e48" containerName="istio-proxy"
Sep 09 12:35:11.737682 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.737653    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="09b7894b-73cb-499d-92ad-d72b0e501e48" containerName="istio-proxy"
Sep 09 12:35:11.737774 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.737721    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="09b7894b-73cb-499d-92ad-d72b0e501e48" containerName="istio-proxy"
Sep 09 12:35:11.737774 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.737730    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="09b7894b-73cb-499d-92ad-d72b0e501e48" containerName="istio-proxy"
Sep 09 12:35:11.741256 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.741234    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.743970 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.743946    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-c356d768-kserve-self-signed-certs\""
Sep 09 12:35:11.751441 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.751418    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"]
Sep 09 12:35:11.846620 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.846585    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-dshm\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.846807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.846631    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-tmp-dir\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.846807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.846726    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-home\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.846807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.846788    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-tls-certs\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.846973 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.846838    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps956\" (UniqueName: \"kubernetes.io/projected/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-kube-api-access-ps956\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.846973 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.846951    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-model-cache\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.947818 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.947784    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-model-cache\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.948004 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.947834    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-dshm\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.948004 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.947890    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-tmp-dir\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.948004 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.947954    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-home\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.948004 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.947986    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-tls-certs\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.948161 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.948036    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps956\" (UniqueName: \"kubernetes.io/projected/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-kube-api-access-ps956\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.948213 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.948178    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-model-cache\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.948319 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.948290    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-tmp-dir\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.948319 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.948305    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-home\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.950284 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.950255    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-dshm\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.950609 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.950591    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-tls-certs\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:11.956713 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:11.956694    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps956\" (UniqueName: \"kubernetes.io/projected/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-kube-api-access-ps956\") pod \"e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") " pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:12.050886 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:12.050836    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:12.325910 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:12.325826    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"]
Sep 09 12:35:12.327359 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:35:12.327321    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22a83f72_7f21_4ff8_8842_1588f7c6f4ae.slice/crio-7370bda52803f025c96108ebae8f26a7c031b183934312e2595a8c10d58e23d9 WatchSource:0}: Error finding container 7370bda52803f025c96108ebae8f26a7c031b183934312e2595a8c10d58e23d9: Status 404 returned error can't find the container with id 7370bda52803f025c96108ebae8f26a7c031b183934312e2595a8c10d58e23d9
Sep 09 12:35:12.809772 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:12.809732    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh" event={"ID":"22a83f72-7f21-4ff8-8842-1588f7c6f4ae","Type":"ContainerStarted","Data":"eff2f4d23dcbdf0b62737d5adee46a89f2dbaa8c78f5ee76ca808c7b8b02d3fb"}
Sep 09 12:35:12.809772 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:12.809780    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh" event={"ID":"22a83f72-7f21-4ff8-8842-1588f7c6f4ae","Type":"ContainerStarted","Data":"7370bda52803f025c96108ebae8f26a7c031b183934312e2595a8c10d58e23d9"}
Sep 09 12:35:12.826528 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:12.826481    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh" podStartSLOduration=1.826459319 podStartE2EDuration="1.826459319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:35:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:35:12.826093053 +0000 UTC m=+932.522658205" watchObservedRunningTime="2026-09-09 12:35:12.826459319 +0000 UTC m=+932.523024484"
Sep 09 12:35:17.078146 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.078109    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"]
Sep 09 12:35:17.082395 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.082374    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.084419 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.084394    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-ace3fa44-openshift-default-dockercfg-ccvj2\""
Sep 09 12:35:17.092491 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.092469    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"]
Sep 09 12:35:17.202470 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.202429    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-credential-socket\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.202717 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.202694    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-token\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.202819 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.202732    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-envoy\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.202906 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.202881    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mql6j\" (UniqueName: \"kubernetes.io/projected/e71de4a3-3e2b-4183-87e6-2979b6df4bee-kube-api-access-mql6j\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.203030 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.203000    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-podinfo\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.203090 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.203037    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-data\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.203209 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.203188    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istiod-ca-cert\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.203273 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.203258    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-workload-certs\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.203327 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.203311    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-workload-socket\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.304317 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.304278    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-workload-certs\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.304317 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.304321    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-workload-socket\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.304660 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.304345    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-credential-socket\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.304660 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.304367    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-token\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.304660 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.304391    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-envoy\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.304660 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.304577    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mql6j\" (UniqueName: \"kubernetes.io/projected/e71de4a3-3e2b-4183-87e6-2979b6df4bee-kube-api-access-mql6j\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.304660 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.304631    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-workload-certs\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.304978 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.304683    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-workload-socket\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.304978 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.304723    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-podinfo\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.304978 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.304745    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-credential-socket\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.304978 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.304754    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-data\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.304978 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.304818    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istiod-ca-cert\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.305264 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.305069    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-data\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.305402 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.305383    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istiod-ca-cert\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.306932 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.306909    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-envoy\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.307313 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.307292    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-podinfo\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.313996 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.313974    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-token\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.314309 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.314287    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mql6j\" (UniqueName: \"kubernetes.io/projected/e71de4a3-3e2b-4183-87e6-2979b6df4bee-kube-api-access-mql6j\") pod \"e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") " pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.395632 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.395547    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:17.675133 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.675104    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"]
Sep 09 12:35:17.676622 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:35:17.676598    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode71de4a3_3e2b_4183_87e6_2979b6df4bee.slice/crio-7ed5fd56768ab119d9f5beaf53c74db90c04c947e70bcd74a2028b7348790b3a WatchSource:0}: Error finding container 7ed5fd56768ab119d9f5beaf53c74db90c04c947e70bcd74a2028b7348790b3a: Status 404 returned error can't find the container with id 7ed5fd56768ab119d9f5beaf53c74db90c04c947e70bcd74a2028b7348790b3a
Sep 09 12:35:17.679155 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.679124    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:35:17.679231 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.679197    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:35:17.679231 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.679226    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:35:17.844043 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.844001    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks" event={"ID":"e71de4a3-3e2b-4183-87e6-2979b6df4bee","Type":"ContainerStarted","Data":"79c92391541e0c98e11ea7fe616bca0e53a059f9afbd6ff25540ec9aa7bc066c"}
Sep 09 12:35:17.844221 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.844085    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks" event={"ID":"e71de4a3-3e2b-4183-87e6-2979b6df4bee","Type":"ContainerStarted","Data":"7ed5fd56768ab119d9f5beaf53c74db90c04c947e70bcd74a2028b7348790b3a"}
Sep 09 12:35:17.864513 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:17.864459    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks" podStartSLOduration=0.864442689 podStartE2EDuration="864.442689ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:35:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:35:17.862103768 +0000 UTC m=+937.558668919" watchObservedRunningTime="2026-09-09 12:35:17.864442689 +0000 UTC m=+937.561007841"
Sep 09 12:35:18.396152 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:18.396113    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:18.401357 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:18.401331    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:18.850712 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:18.850683    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:18.851624 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:18.851601    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:22.051173 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:22.051137    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:22.051613 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:22.051184    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:22.064384 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:22.064359    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:22.889186 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:22.889151    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:35:40.993270 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:40.993234    2582 generic.go:356] "Generic (PLEG): container finished" podID="e71de4a3-3e2b-4183-87e6-2979b6df4bee" containerID="79c92391541e0c98e11ea7fe616bca0e53a059f9afbd6ff25540ec9aa7bc066c" exitCode=0
Sep 09 12:35:40.993640 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:40.993304    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks" event={"ID":"e71de4a3-3e2b-4183-87e6-2979b6df4bee","Type":"ContainerDied","Data":"79c92391541e0c98e11ea7fe616bca0e53a059f9afbd6ff25540ec9aa7bc066c"}
Sep 09 12:35:40.993705 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:40.993678    2582 scope.go:117] "RemoveContainer" containerID="79c92391541e0c98e11ea7fe616bca0e53a059f9afbd6ff25540ec9aa7bc066c"
Sep 09 12:35:40.994826 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:40.994795    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:35:40.994931 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:40.994889    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:35:40.994985 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:40.994932    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:35:42.003898 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:42.003863    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks" event={"ID":"e71de4a3-3e2b-4183-87e6-2979b6df4bee","Type":"ContainerStarted","Data":"fe0d64b66fb7230d594c52b4eec8313c7cfee403d9a48503aeb8508c42d4b1d0"}
Sep 09 12:35:42.396043 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:42.395942    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:42.396230 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:42.396184    2582 patch_prober.go:28] interesting pod/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 09 12:35:42.396305 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:42.396253    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks" podUID="e71de4a3-3e2b-4183-87e6-2979b6df4bee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 09 12:35:43.396150 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:43.396111    2582 patch_prober.go:28] interesting pod/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 09 12:35:43.396614 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:43.396176    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks" podUID="e71de4a3-3e2b-4183-87e6-2979b6df4bee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 09 12:35:44.396571 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:44.396526    2582 patch_prober.go:28] interesting pod/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 09 12:35:44.397055 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:44.396600    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks" podUID="e71de4a3-3e2b-4183-87e6-2979b6df4bee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 09 12:35:45.397115 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:45.397064    2582 patch_prober.go:28] interesting pod/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 09 12:35:45.397632 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:45.397144    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks" podUID="e71de4a3-3e2b-4183-87e6-2979b6df4bee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 09 12:35:46.396126 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:46.396090    2582 patch_prober.go:28] interesting pod/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 09 12:35:46.396425 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:46.396387    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks" podUID="e71de4a3-3e2b-4183-87e6-2979b6df4bee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 09 12:35:47.400478 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:47.400448    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:47.400944 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:47.400899    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:47.401678 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:47.401658    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:51.401873 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:51.401693    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"]
Sep 09 12:35:51.402433 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:51.401979    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks" podUID="e71de4a3-3e2b-4183-87e6-2979b6df4bee" containerName="istio-proxy" containerID="cri-o://fe0d64b66fb7230d594c52b4eec8313c7cfee403d9a48503aeb8508c42d4b1d0" gracePeriod=30
Sep 09 12:35:56.670221 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.670153    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:56.835056 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835024    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istiod-ca-cert\") pod \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") "
Sep 09 12:35:56.835056 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835057    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-podinfo\") pod \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") "
Sep 09 12:35:56.835283 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835077    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-credential-socket\") pod \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") "
Sep 09 12:35:56.835283 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835111    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-token\") pod \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") "
Sep 09 12:35:56.835283 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835148    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-workload-socket\") pod \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") "
Sep 09 12:35:56.835283 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835193    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mql6j\" (UniqueName: \"kubernetes.io/projected/e71de4a3-3e2b-4183-87e6-2979b6df4bee-kube-api-access-mql6j\") pod \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") "
Sep 09 12:35:56.835283 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835246    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-envoy\") pod \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") "
Sep 09 12:35:56.835283 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835264    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-data\") pod \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") "
Sep 09 12:35:56.835615 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835303    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-workload-certs\") pod \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\" (UID: \"e71de4a3-3e2b-4183-87e6-2979b6df4bee\") "
Sep 09 12:35:56.835615 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835344    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e71de4a3-3e2b-4183-87e6-2979b6df4bee" (UID: "e71de4a3-3e2b-4183-87e6-2979b6df4bee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:56.835615 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835465    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e71de4a3-3e2b-4183-87e6-2979b6df4bee" (UID: "e71de4a3-3e2b-4183-87e6-2979b6df4bee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:35:56.835615 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835566    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e71de4a3-3e2b-4183-87e6-2979b6df4bee" (UID: "e71de4a3-3e2b-4183-87e6-2979b6df4bee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:56.835615 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835578    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-data" (OuterVolumeSpecName: "istio-data") pod "e71de4a3-3e2b-4183-87e6-2979b6df4bee" (UID: "e71de4a3-3e2b-4183-87e6-2979b6df4bee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:56.836069 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835662    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e71de4a3-3e2b-4183-87e6-2979b6df4bee" (UID: "e71de4a3-3e2b-4183-87e6-2979b6df4bee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:56.836069 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835775    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:35:56.836069 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835792    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-workload-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:35:56.836069 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835805    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istiod-ca-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:35:56.836069 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835817    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-credential-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:35:56.836069 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.835832    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-workload-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:35:56.837543 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.837511    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e71de4a3-3e2b-4183-87e6-2979b6df4bee-kube-api-access-mql6j" (OuterVolumeSpecName: "kube-api-access-mql6j") pod "e71de4a3-3e2b-4183-87e6-2979b6df4bee" (UID: "e71de4a3-3e2b-4183-87e6-2979b6df4bee"). InnerVolumeSpecName "kube-api-access-mql6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:35:56.837543 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.837530    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-token" (OuterVolumeSpecName: "istio-token") pod "e71de4a3-3e2b-4183-87e6-2979b6df4bee" (UID: "e71de4a3-3e2b-4183-87e6-2979b6df4bee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:35:56.837692 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.837557    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e71de4a3-3e2b-4183-87e6-2979b6df4bee" (UID: "e71de4a3-3e2b-4183-87e6-2979b6df4bee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:35:56.837692 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.837623    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e71de4a3-3e2b-4183-87e6-2979b6df4bee" (UID: "e71de4a3-3e2b-4183-87e6-2979b6df4bee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:56.936513 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.936435    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mql6j\" (UniqueName: \"kubernetes.io/projected/e71de4a3-3e2b-4183-87e6-2979b6df4bee-kube-api-access-mql6j\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:35:56.936513 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.936462    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-envoy\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:35:56.936513 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.936473    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-podinfo\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:35:56.936513 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:56.936480    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e71de4a3-3e2b-4183-87e6-2979b6df4bee-istio-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:35:57.107557 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:57.107522    2582 generic.go:356] "Generic (PLEG): container finished" podID="e71de4a3-3e2b-4183-87e6-2979b6df4bee" containerID="fe0d64b66fb7230d594c52b4eec8313c7cfee403d9a48503aeb8508c42d4b1d0" exitCode=0
Sep 09 12:35:57.107742 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:57.107595    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"
Sep 09 12:35:57.107742 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:57.107611    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks" event={"ID":"e71de4a3-3e2b-4183-87e6-2979b6df4bee","Type":"ContainerDied","Data":"fe0d64b66fb7230d594c52b4eec8313c7cfee403d9a48503aeb8508c42d4b1d0"}
Sep 09 12:35:57.107742 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:57.107691    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks" event={"ID":"e71de4a3-3e2b-4183-87e6-2979b6df4bee","Type":"ContainerDied","Data":"7ed5fd56768ab119d9f5beaf53c74db90c04c947e70bcd74a2028b7348790b3a"}
Sep 09 12:35:57.107742 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:57.107713    2582 scope.go:117] "RemoveContainer" containerID="fe0d64b66fb7230d594c52b4eec8313c7cfee403d9a48503aeb8508c42d4b1d0"
Sep 09 12:35:57.118088 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:57.118066    2582 scope.go:117] "RemoveContainer" containerID="79c92391541e0c98e11ea7fe616bca0e53a059f9afbd6ff25540ec9aa7bc066c"
Sep 09 12:35:57.127409 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:57.127388    2582 scope.go:117] "RemoveContainer" containerID="fe0d64b66fb7230d594c52b4eec8313c7cfee403d9a48503aeb8508c42d4b1d0"
Sep 09 12:35:57.127753 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:35:57.127724    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe0d64b66fb7230d594c52b4eec8313c7cfee403d9a48503aeb8508c42d4b1d0\": container with ID starting with fe0d64b66fb7230d594c52b4eec8313c7cfee403d9a48503aeb8508c42d4b1d0 not found: ID does not exist" containerID="fe0d64b66fb7230d594c52b4eec8313c7cfee403d9a48503aeb8508c42d4b1d0"
Sep 09 12:35:57.127875 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:57.127759    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe0d64b66fb7230d594c52b4eec8313c7cfee403d9a48503aeb8508c42d4b1d0"} err="failed to get container status \"fe0d64b66fb7230d594c52b4eec8313c7cfee403d9a48503aeb8508c42d4b1d0\": rpc error: code = NotFound desc = could not find container \"fe0d64b66fb7230d594c52b4eec8313c7cfee403d9a48503aeb8508c42d4b1d0\": container with ID starting with fe0d64b66fb7230d594c52b4eec8313c7cfee403d9a48503aeb8508c42d4b1d0 not found: ID does not exist"
Sep 09 12:35:57.127875 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:57.127783    2582 scope.go:117] "RemoveContainer" containerID="79c92391541e0c98e11ea7fe616bca0e53a059f9afbd6ff25540ec9aa7bc066c"
Sep 09 12:35:57.128159 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:35:57.128140    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79c92391541e0c98e11ea7fe616bca0e53a059f9afbd6ff25540ec9aa7bc066c\": container with ID starting with 79c92391541e0c98e11ea7fe616bca0e53a059f9afbd6ff25540ec9aa7bc066c not found: ID does not exist" containerID="79c92391541e0c98e11ea7fe616bca0e53a059f9afbd6ff25540ec9aa7bc066c"
Sep 09 12:35:57.128260 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:57.128164    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79c92391541e0c98e11ea7fe616bca0e53a059f9afbd6ff25540ec9aa7bc066c"} err="failed to get container status \"79c92391541e0c98e11ea7fe616bca0e53a059f9afbd6ff25540ec9aa7bc066c\": rpc error: code = NotFound desc = could not find container \"79c92391541e0c98e11ea7fe616bca0e53a059f9afbd6ff25540ec9aa7bc066c\": container with ID starting with 79c92391541e0c98e11ea7fe616bca0e53a059f9afbd6ff25540ec9aa7bc066c not found: ID does not exist"
Sep 09 12:35:57.130659 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:57.130636    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"]
Sep 09 12:35:57.133248 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:57.133226    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-ace3fa44-openshift-default-999c5f4d8-2p6ks"]
Sep 09 12:35:58.825335 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:35:58.825302    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e71de4a3-3e2b-4183-87e6-2979b6df4bee" path="/var/lib/kubelet/pods/e71de4a3-3e2b-4183-87e6-2979b6df4bee/volumes"
Sep 09 12:36:16.220542 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:16.220451    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"]
Sep 09 12:36:27.621136 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:27.621101    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"]
Sep 09 12:36:27.621730 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:27.621327    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h" podUID="15521e6b-3cc6-44a8-9e54-e8bda208836e" containerName="maas-api" containerID="cri-o://bdb9d7d3fac454b7f1f5e2fe88a6cda251749151ab24f0b43156e40379b6ecf1" gracePeriod=30
Sep 09 12:36:27.889757 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:27.889732    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:36:28.033871 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.033817    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjdvd\" (UniqueName: \"kubernetes.io/projected/15521e6b-3cc6-44a8-9e54-e8bda208836e-kube-api-access-fjdvd\") pod \"15521e6b-3cc6-44a8-9e54-e8bda208836e\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") "
Sep 09 12:36:28.034071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.033949    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15521e6b-3cc6-44a8-9e54-e8bda208836e-openshift-service-ca\") pod \"15521e6b-3cc6-44a8-9e54-e8bda208836e\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") "
Sep 09 12:36:28.034071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.034047    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15521e6b-3cc6-44a8-9e54-e8bda208836e-maas-api-tls\") pod \"15521e6b-3cc6-44a8-9e54-e8bda208836e\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") "
Sep 09 12:36:28.034171 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.034104    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15521e6b-3cc6-44a8-9e54-e8bda208836e-metrics-certs\") pod \"15521e6b-3cc6-44a8-9e54-e8bda208836e\" (UID: \"15521e6b-3cc6-44a8-9e54-e8bda208836e\") "
Sep 09 12:36:28.034694 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.034667    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15521e6b-3cc6-44a8-9e54-e8bda208836e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "15521e6b-3cc6-44a8-9e54-e8bda208836e" (UID: "15521e6b-3cc6-44a8-9e54-e8bda208836e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:36:28.039321 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.039285    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15521e6b-3cc6-44a8-9e54-e8bda208836e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "15521e6b-3cc6-44a8-9e54-e8bda208836e" (UID: "15521e6b-3cc6-44a8-9e54-e8bda208836e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:36:28.039804 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.039762    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15521e6b-3cc6-44a8-9e54-e8bda208836e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "15521e6b-3cc6-44a8-9e54-e8bda208836e" (UID: "15521e6b-3cc6-44a8-9e54-e8bda208836e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:36:28.043017 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.042980    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15521e6b-3cc6-44a8-9e54-e8bda208836e-kube-api-access-fjdvd" (OuterVolumeSpecName: "kube-api-access-fjdvd") pod "15521e6b-3cc6-44a8-9e54-e8bda208836e" (UID: "15521e6b-3cc6-44a8-9e54-e8bda208836e"). InnerVolumeSpecName "kube-api-access-fjdvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:36:28.135748 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.135679    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjdvd\" (UniqueName: \"kubernetes.io/projected/15521e6b-3cc6-44a8-9e54-e8bda208836e-kube-api-access-fjdvd\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:28.135748 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.135705    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15521e6b-3cc6-44a8-9e54-e8bda208836e-openshift-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:28.135748 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.135715    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15521e6b-3cc6-44a8-9e54-e8bda208836e-maas-api-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:28.135748 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.135723    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15521e6b-3cc6-44a8-9e54-e8bda208836e-metrics-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:28.303410 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.303379    2582 generic.go:356] "Generic (PLEG): container finished" podID="15521e6b-3cc6-44a8-9e54-e8bda208836e" containerID="bdb9d7d3fac454b7f1f5e2fe88a6cda251749151ab24f0b43156e40379b6ecf1" exitCode=0
Sep 09 12:36:28.303564 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.303425    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"
Sep 09 12:36:28.303564 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.303472    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h" event={"ID":"15521e6b-3cc6-44a8-9e54-e8bda208836e","Type":"ContainerDied","Data":"bdb9d7d3fac454b7f1f5e2fe88a6cda251749151ab24f0b43156e40379b6ecf1"}
Sep 09 12:36:28.303564 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.303517    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h" event={"ID":"15521e6b-3cc6-44a8-9e54-e8bda208836e","Type":"ContainerDied","Data":"8185dabf953b82f67b34fda3aca7a2aa6c548a4155265db879ca7df6d68eed33"}
Sep 09 12:36:28.303564 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.303539    2582 scope.go:117] "RemoveContainer" containerID="bdb9d7d3fac454b7f1f5e2fe88a6cda251749151ab24f0b43156e40379b6ecf1"
Sep 09 12:36:28.313879 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.313864    2582 scope.go:117] "RemoveContainer" containerID="bdb9d7d3fac454b7f1f5e2fe88a6cda251749151ab24f0b43156e40379b6ecf1"
Sep 09 12:36:28.314165 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:36:28.314145    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdb9d7d3fac454b7f1f5e2fe88a6cda251749151ab24f0b43156e40379b6ecf1\": container with ID starting with bdb9d7d3fac454b7f1f5e2fe88a6cda251749151ab24f0b43156e40379b6ecf1 not found: ID does not exist" containerID="bdb9d7d3fac454b7f1f5e2fe88a6cda251749151ab24f0b43156e40379b6ecf1"
Sep 09 12:36:28.314237 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.314172    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdb9d7d3fac454b7f1f5e2fe88a6cda251749151ab24f0b43156e40379b6ecf1"} err="failed to get container status \"bdb9d7d3fac454b7f1f5e2fe88a6cda251749151ab24f0b43156e40379b6ecf1\": rpc error: code = NotFound desc = could not find container \"bdb9d7d3fac454b7f1f5e2fe88a6cda251749151ab24f0b43156e40379b6ecf1\": container with ID starting with bdb9d7d3fac454b7f1f5e2fe88a6cda251749151ab24f0b43156e40379b6ecf1 not found: ID does not exist"
Sep 09 12:36:28.324934 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.324910    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"]
Sep 09 12:36:28.327570 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.327548    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e917b0-5bc95fbcc-p2z2h"]
Sep 09 12:36:28.825233 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:28.825200    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15521e6b-3cc6-44a8-9e54-e8bda208836e" path="/var/lib/kubelet/pods/15521e6b-3cc6-44a8-9e54-e8bda208836e/volumes"
Sep 09 12:36:31.244726 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.244666    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh" podUID="22a83f72-7f21-4ff8-8842-1588f7c6f4ae" containerName="main" containerID="cri-o://eff2f4d23dcbdf0b62737d5adee46a89f2dbaa8c78f5ee76ca808c7b8b02d3fb" gracePeriod=45
Sep 09 12:36:31.509564 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.509504    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:36:31.674130 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.674101    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-tmp-dir\") pod \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") "
Sep 09 12:36:31.674275 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.674163    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-tls-certs\") pod \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") "
Sep 09 12:36:31.674275 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.674202    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-home\") pod \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") "
Sep 09 12:36:31.674275 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.674234    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-model-cache\") pod \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") "
Sep 09 12:36:31.674460 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.674275    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-dshm\") pod \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") "
Sep 09 12:36:31.674460 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.674302    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ps956\" (UniqueName: \"kubernetes.io/projected/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-kube-api-access-ps956\") pod \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\" (UID: \"22a83f72-7f21-4ff8-8842-1588f7c6f4ae\") "
Sep 09 12:36:31.674460 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.674415    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "22a83f72-7f21-4ff8-8842-1588f7c6f4ae" (UID: "22a83f72-7f21-4ff8-8842-1588f7c6f4ae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:36:31.674460 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.674451    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-home" (OuterVolumeSpecName: "home") pod "22a83f72-7f21-4ff8-8842-1588f7c6f4ae" (UID: "22a83f72-7f21-4ff8-8842-1588f7c6f4ae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:36:31.674657 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.674496    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-model-cache" (OuterVolumeSpecName: "model-cache") pod "22a83f72-7f21-4ff8-8842-1588f7c6f4ae" (UID: "22a83f72-7f21-4ff8-8842-1588f7c6f4ae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:36:31.674727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.674694    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-home\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:31.674727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.674713    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-model-cache\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:31.674803 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.674729    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-tmp-dir\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:31.676451 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.676418    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "22a83f72-7f21-4ff8-8842-1588f7c6f4ae" (UID: "22a83f72-7f21-4ff8-8842-1588f7c6f4ae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:36:31.676567 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.676468    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-dshm" (OuterVolumeSpecName: "dshm") pod "22a83f72-7f21-4ff8-8842-1588f7c6f4ae" (UID: "22a83f72-7f21-4ff8-8842-1588f7c6f4ae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:36:31.676567 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.676480    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-kube-api-access-ps956" (OuterVolumeSpecName: "kube-api-access-ps956") pod "22a83f72-7f21-4ff8-8842-1588f7c6f4ae" (UID: "22a83f72-7f21-4ff8-8842-1588f7c6f4ae"). InnerVolumeSpecName "kube-api-access-ps956". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:36:31.776202 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.776137    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-tls-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:31.776202 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.776163    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-dshm\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:31.776202 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:31.776171    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ps956\" (UniqueName: \"kubernetes.io/projected/22a83f72-7f21-4ff8-8842-1588f7c6f4ae-kube-api-access-ps956\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:32.330116 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:32.330075    2582 generic.go:356] "Generic (PLEG): container finished" podID="22a83f72-7f21-4ff8-8842-1588f7c6f4ae" containerID="eff2f4d23dcbdf0b62737d5adee46a89f2dbaa8c78f5ee76ca808c7b8b02d3fb" exitCode=0
Sep 09 12:36:32.330510 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:32.330125    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"
Sep 09 12:36:32.330510 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:32.330158    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh" event={"ID":"22a83f72-7f21-4ff8-8842-1588f7c6f4ae","Type":"ContainerDied","Data":"eff2f4d23dcbdf0b62737d5adee46a89f2dbaa8c78f5ee76ca808c7b8b02d3fb"}
Sep 09 12:36:32.330510 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:32.330200    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh" event={"ID":"22a83f72-7f21-4ff8-8842-1588f7c6f4ae","Type":"ContainerDied","Data":"7370bda52803f025c96108ebae8f26a7c031b183934312e2595a8c10d58e23d9"}
Sep 09 12:36:32.330510 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:32.330218    2582 scope.go:117] "RemoveContainer" containerID="eff2f4d23dcbdf0b62737d5adee46a89f2dbaa8c78f5ee76ca808c7b8b02d3fb"
Sep 09 12:36:32.340122 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:32.340106    2582 scope.go:117] "RemoveContainer" containerID="eff2f4d23dcbdf0b62737d5adee46a89f2dbaa8c78f5ee76ca808c7b8b02d3fb"
Sep 09 12:36:32.340390 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:36:32.340372    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eff2f4d23dcbdf0b62737d5adee46a89f2dbaa8c78f5ee76ca808c7b8b02d3fb\": container with ID starting with eff2f4d23dcbdf0b62737d5adee46a89f2dbaa8c78f5ee76ca808c7b8b02d3fb not found: ID does not exist" containerID="eff2f4d23dcbdf0b62737d5adee46a89f2dbaa8c78f5ee76ca808c7b8b02d3fb"
Sep 09 12:36:32.340438 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:32.340399    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eff2f4d23dcbdf0b62737d5adee46a89f2dbaa8c78f5ee76ca808c7b8b02d3fb"} err="failed to get container status \"eff2f4d23dcbdf0b62737d5adee46a89f2dbaa8c78f5ee76ca808c7b8b02d3fb\": rpc error: code = NotFound desc = could not find container \"eff2f4d23dcbdf0b62737d5adee46a89f2dbaa8c78f5ee76ca808c7b8b02d3fb\": container with ID starting with eff2f4d23dcbdf0b62737d5adee46a89f2dbaa8c78f5ee76ca808c7b8b02d3fb not found: ID does not exist"
Sep 09 12:36:32.352652 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:32.352631    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"]
Sep 09 12:36:32.355262 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:32.355238    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-c356d768-kserve-5b6bfbf68f-pzgwh"]
Sep 09 12:36:32.826250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:32.826220    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22a83f72-7f21-4ff8-8842-1588f7c6f4ae" path="/var/lib/kubelet/pods/22a83f72-7f21-4ff8-8842-1588f7c6f4ae/volumes"
Sep 09 12:36:35.097079 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:35.097045    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"]
Sep 09 12:36:35.097562 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:35.097278    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c" podUID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" containerName="istio-proxy" containerID="cri-o://1e3d77f449a70d2cd7bda90e2a6d72737aeb5742a41b7786058044076e48af64" gracePeriod=30
Sep 09 12:36:40.393292 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.393214    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:36:40.395370 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.395337    2582 generic.go:356] "Generic (PLEG): container finished" podID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" containerID="1e3d77f449a70d2cd7bda90e2a6d72737aeb5742a41b7786058044076e48af64" exitCode=0
Sep 09 12:36:40.395500 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.395405    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c" event={"ID":"63c0d03c-3936-49e2-9c39-d5328cf5e99b","Type":"ContainerDied","Data":"1e3d77f449a70d2cd7bda90e2a6d72737aeb5742a41b7786058044076e48af64"}
Sep 09 12:36:40.395549 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.395511    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c" event={"ID":"63c0d03c-3936-49e2-9c39-d5328cf5e99b","Type":"ContainerDied","Data":"0547ddf2a106285dfa45286c155afe52dbcc175b5ddaabc14d1a6111f73ff5b9"}
Sep 09 12:36:40.395549 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.395540    2582 scope.go:117] "RemoveContainer" containerID="1e3d77f449a70d2cd7bda90e2a6d72737aeb5742a41b7786058044076e48af64"
Sep 09 12:36:40.405230 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.405209    2582 scope.go:117] "RemoveContainer" containerID="f8b79e66c3be91f2e56d0a2f70992fd00265826f2b86e620aea3a70e294201a9"
Sep 09 12:36:40.414264 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.414244    2582 scope.go:117] "RemoveContainer" containerID="1e3d77f449a70d2cd7bda90e2a6d72737aeb5742a41b7786058044076e48af64"
Sep 09 12:36:40.414566 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:36:40.414542    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e3d77f449a70d2cd7bda90e2a6d72737aeb5742a41b7786058044076e48af64\": container with ID starting with 1e3d77f449a70d2cd7bda90e2a6d72737aeb5742a41b7786058044076e48af64 not found: ID does not exist" containerID="1e3d77f449a70d2cd7bda90e2a6d72737aeb5742a41b7786058044076e48af64"
Sep 09 12:36:40.414613 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.414579    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e3d77f449a70d2cd7bda90e2a6d72737aeb5742a41b7786058044076e48af64"} err="failed to get container status \"1e3d77f449a70d2cd7bda90e2a6d72737aeb5742a41b7786058044076e48af64\": rpc error: code = NotFound desc = could not find container \"1e3d77f449a70d2cd7bda90e2a6d72737aeb5742a41b7786058044076e48af64\": container with ID starting with 1e3d77f449a70d2cd7bda90e2a6d72737aeb5742a41b7786058044076e48af64 not found: ID does not exist"
Sep 09 12:36:40.414613 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.414599    2582 scope.go:117] "RemoveContainer" containerID="f8b79e66c3be91f2e56d0a2f70992fd00265826f2b86e620aea3a70e294201a9"
Sep 09 12:36:40.415385 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:36:40.415232    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8b79e66c3be91f2e56d0a2f70992fd00265826f2b86e620aea3a70e294201a9\": container with ID starting with f8b79e66c3be91f2e56d0a2f70992fd00265826f2b86e620aea3a70e294201a9 not found: ID does not exist" containerID="f8b79e66c3be91f2e56d0a2f70992fd00265826f2b86e620aea3a70e294201a9"
Sep 09 12:36:40.415385 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.415265    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8b79e66c3be91f2e56d0a2f70992fd00265826f2b86e620aea3a70e294201a9"} err="failed to get container status \"f8b79e66c3be91f2e56d0a2f70992fd00265826f2b86e620aea3a70e294201a9\": rpc error: code = NotFound desc = could not find container \"f8b79e66c3be91f2e56d0a2f70992fd00265826f2b86e620aea3a70e294201a9\": container with ID starting with f8b79e66c3be91f2e56d0a2f70992fd00265826f2b86e620aea3a70e294201a9 not found: ID does not exist"
Sep 09 12:36:40.453924 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.453884    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-workload-certs\") pod \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") "
Sep 09 12:36:40.454096 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.453946    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-podinfo\") pod \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") "
Sep 09 12:36:40.454096 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.453989    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-token\") pod \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") "
Sep 09 12:36:40.454096 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.454039    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-workload-socket\") pod \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") "
Sep 09 12:36:40.454096 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.454067    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istiod-ca-cert\") pod \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") "
Sep 09 12:36:40.454322 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.454103    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjjj7\" (UniqueName: \"kubernetes.io/projected/63c0d03c-3936-49e2-9c39-d5328cf5e99b-kube-api-access-kjjj7\") pod \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") "
Sep 09 12:36:40.454322 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.454138    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-credential-socket\") pod \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") "
Sep 09 12:36:40.454322 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.454147    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "63c0d03c-3936-49e2-9c39-d5328cf5e99b" (UID: "63c0d03c-3936-49e2-9c39-d5328cf5e99b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:36:40.454322 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.454215    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-envoy\") pod \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") "
Sep 09 12:36:40.454322 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.454304    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-data\") pod \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\" (UID: \"63c0d03c-3936-49e2-9c39-d5328cf5e99b\") "
Sep 09 12:36:40.454582 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.454568    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-workload-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:40.454640 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.454599    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "63c0d03c-3936-49e2-9c39-d5328cf5e99b" (UID: "63c0d03c-3936-49e2-9c39-d5328cf5e99b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:36:40.454640 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.454513    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "63c0d03c-3936-49e2-9c39-d5328cf5e99b" (UID: "63c0d03c-3936-49e2-9c39-d5328cf5e99b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:36:40.454912 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.454644    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "63c0d03c-3936-49e2-9c39-d5328cf5e99b" (UID: "63c0d03c-3936-49e2-9c39-d5328cf5e99b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:36:40.454912 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.454743    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-data" (OuterVolumeSpecName: "istio-data") pod "63c0d03c-3936-49e2-9c39-d5328cf5e99b" (UID: "63c0d03c-3936-49e2-9c39-d5328cf5e99b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:36:40.456520 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.456480    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-token" (OuterVolumeSpecName: "istio-token") pod "63c0d03c-3936-49e2-9c39-d5328cf5e99b" (UID: "63c0d03c-3936-49e2-9c39-d5328cf5e99b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:36:40.456646 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.456557    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63c0d03c-3936-49e2-9c39-d5328cf5e99b-kube-api-access-kjjj7" (OuterVolumeSpecName: "kube-api-access-kjjj7") pod "63c0d03c-3936-49e2-9c39-d5328cf5e99b" (UID: "63c0d03c-3936-49e2-9c39-d5328cf5e99b"). InnerVolumeSpecName "kube-api-access-kjjj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:36:40.456646 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.456615    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "63c0d03c-3936-49e2-9c39-d5328cf5e99b" (UID: "63c0d03c-3936-49e2-9c39-d5328cf5e99b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:36:40.456760 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.456688    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "63c0d03c-3936-49e2-9c39-d5328cf5e99b" (UID: "63c0d03c-3936-49e2-9c39-d5328cf5e99b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:36:40.555036 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.555004    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-podinfo\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:40.555036 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.555038    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:40.555272 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.555052    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-workload-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:40.555272 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.555062    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istiod-ca-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:40.555272 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.555072    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjjj7\" (UniqueName: \"kubernetes.io/projected/63c0d03c-3936-49e2-9c39-d5328cf5e99b-kube-api-access-kjjj7\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:40.555272 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.555084    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-credential-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:40.555272 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.555097    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-envoy\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:40.555272 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:40.555108    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63c0d03c-3936-49e2-9c39-d5328cf5e99b-istio-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:36:41.402142 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:41.402113    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"
Sep 09 12:36:41.437052 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:41.437023    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"]
Sep 09 12:36:41.444744 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:41.444715    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-e917b0-openshift-default-5d8cb6698b-vdq8c"]
Sep 09 12:36:42.825318 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:42.825285    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" path="/var/lib/kubelet/pods/63c0d03c-3936-49e2-9c39-d5328cf5e99b/volumes"
Sep 09 12:36:50.611423 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.611382    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"]
Sep 09 12:36:50.612079 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612057    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e71de4a3-3e2b-4183-87e6-2979b6df4bee" containerName="istio-proxy"
Sep 09 12:36:50.612079 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612077    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e71de4a3-3e2b-4183-87e6-2979b6df4bee" containerName="istio-proxy"
Sep 09 12:36:50.612213 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612092    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" containerName="istio-proxy"
Sep 09 12:36:50.612213 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612101    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" containerName="istio-proxy"
Sep 09 12:36:50.612213 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612111    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e71de4a3-3e2b-4183-87e6-2979b6df4bee" containerName="istio-proxy"
Sep 09 12:36:50.612213 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612121    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e71de4a3-3e2b-4183-87e6-2979b6df4bee" containerName="istio-proxy"
Sep 09 12:36:50.612213 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612134    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" containerName="istio-proxy"
Sep 09 12:36:50.612213 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612142    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" containerName="istio-proxy"
Sep 09 12:36:50.612213 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612154    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15521e6b-3cc6-44a8-9e54-e8bda208836e" containerName="maas-api"
Sep 09 12:36:50.612213 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612162    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="15521e6b-3cc6-44a8-9e54-e8bda208836e" containerName="maas-api"
Sep 09 12:36:50.612213 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612186    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22a83f72-7f21-4ff8-8842-1588f7c6f4ae" containerName="main"
Sep 09 12:36:50.612213 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612194    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="22a83f72-7f21-4ff8-8842-1588f7c6f4ae" containerName="main"
Sep 09 12:36:50.612693 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612299    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="15521e6b-3cc6-44a8-9e54-e8bda208836e" containerName="maas-api"
Sep 09 12:36:50.612693 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612310    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" containerName="istio-proxy"
Sep 09 12:36:50.612693 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612322    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="22a83f72-7f21-4ff8-8842-1588f7c6f4ae" containerName="main"
Sep 09 12:36:50.612693 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612331    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e71de4a3-3e2b-4183-87e6-2979b6df4bee" containerName="istio-proxy"
Sep 09 12:36:50.612693 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.612344    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e71de4a3-3e2b-4183-87e6-2979b6df4bee" containerName="istio-proxy"
Sep 09 12:36:50.617217 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.617197    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.620257 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.620230    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5e5923\"/\"openshift-service-ca.crt\""
Sep 09 12:36:50.620390 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.620258    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5e5923\"/\"default-dockercfg-67c89\""
Sep 09 12:36:50.620390 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.620262    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5e5923\"/\"kube-root-ca.crt\""
Sep 09 12:36:50.620390 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.620261    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5e5923\"/\"test-model-5e5923-kserve-self-signed-certs\""
Sep 09 12:36:50.626170 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.626145    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"]
Sep 09 12:36:50.661576 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.661532    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-home\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.661772 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.661645    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c64328a-8bf0-4b59-bf33-b337780a0773-tls-certs\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.661772 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.661674    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-dshm\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.661772 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.661715    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-tmp-dir\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.662012 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.661787    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-model-cache\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.662012 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.661907    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkpfp\" (UniqueName: \"kubernetes.io/projected/0c64328a-8bf0-4b59-bf33-b337780a0773-kube-api-access-tkpfp\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.763452 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.763397    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c64328a-8bf0-4b59-bf33-b337780a0773-tls-certs\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.763452 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.763456    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-dshm\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.763452 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.763478    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-tmp-dir\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.763788 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.763511    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-model-cache\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.763788 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.763552    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkpfp\" (UniqueName: \"kubernetes.io/projected/0c64328a-8bf0-4b59-bf33-b337780a0773-kube-api-access-tkpfp\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.763788 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.763576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-home\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.764063 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.764032    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-tmp-dir\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.764179 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.764046    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-model-cache\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.764179 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.764101    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-home\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.766057 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.766035    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-dshm\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.766204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.766186    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c64328a-8bf0-4b59-bf33-b337780a0773-tls-certs\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.772111 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.772085    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkpfp\" (UniqueName: \"kubernetes.io/projected/0c64328a-8bf0-4b59-bf33-b337780a0773-kube-api-access-tkpfp\") pod \"test-model-5e5923-kserve-57f9f4bcb7-67b92\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") " pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:50.929240 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:50.929150    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:36:51.208585 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:51.208548    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"]
Sep 09 12:36:51.209586 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:36:51.209547    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c64328a_8bf0_4b59_bf33_b337780a0773.slice/crio-3558ed97382d8ae6b17e8c802d55196dfd1769536defc4396448b09d226ea9d4 WatchSource:0}: Error finding container 3558ed97382d8ae6b17e8c802d55196dfd1769536defc4396448b09d226ea9d4: Status 404 returned error can't find the container with id 3558ed97382d8ae6b17e8c802d55196dfd1769536defc4396448b09d226ea9d4
Sep 09 12:36:51.467366 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:51.467277    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92" event={"ID":"0c64328a-8bf0-4b59-bf33-b337780a0773","Type":"ContainerStarted","Data":"1466ab38c2a37ebd966531d20b93d16ec0959b9ccf476a85e8043358d2a6489d"}
Sep 09 12:36:51.467366 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:51.467318    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92" event={"ID":"0c64328a-8bf0-4b59-bf33-b337780a0773","Type":"ContainerStarted","Data":"3558ed97382d8ae6b17e8c802d55196dfd1769536defc4396448b09d226ea9d4"}
Sep 09 12:36:51.484063 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:36:51.484009    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92" podStartSLOduration=1.483990733 podStartE2EDuration="1.483990733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:36:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:36:51.482927964 +0000 UTC m=+1031.179493115" watchObservedRunningTime="2026-09-09 12:36:51.483990733 +0000 UTC m=+1031.180555887"
Sep 09 12:37:00.929894 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:00.929842    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:37:00.930332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:00.929915    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:37:00.944041 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:00.944018    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:37:01.544076 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:01.544049    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:37:21.618935 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.618907    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"]
Sep 09 12:37:21.619397 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.619384    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="63c0d03c-3936-49e2-9c39-d5328cf5e99b" containerName="istio-proxy"
Sep 09 12:37:21.622662 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.622641    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.624737 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.624715    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-0f93fa\"/\"kube-root-ca.crt\""
Sep 09 12:37:21.625439 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.625413    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-0f93fa\"/\"default-dockercfg-6pjtx\""
Sep 09 12:37:21.625566 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.625443    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-0f93fa\"/\"explicit-model-0f93fa-kserve-self-signed-certs\""
Sep 09 12:37:21.625680 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.625652    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-0f93fa\"/\"openshift-service-ca.crt\""
Sep 09 12:37:21.633923 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.633886    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"]
Sep 09 12:37:21.649384 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.649354    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-model-cache\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.649543 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.649434    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-home\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.649543 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.649469    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8j7m\" (UniqueName: \"kubernetes.io/projected/de3070b8-12ff-4726-b3b2-499d758ba5fa-kube-api-access-k8j7m\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.649543 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.649506    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-dshm\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.649543 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.649540    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3070b8-12ff-4726-b3b2-499d758ba5fa-tls-certs\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.649798 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.649674    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-tmp-dir\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.750579 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.750544    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-home\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.750579 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.750585    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8j7m\" (UniqueName: \"kubernetes.io/projected/de3070b8-12ff-4726-b3b2-499d758ba5fa-kube-api-access-k8j7m\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.750934 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.750617    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-dshm\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.750934 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.750647    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3070b8-12ff-4726-b3b2-499d758ba5fa-tls-certs\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.750934 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.750700    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-tmp-dir\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.750934 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.750812    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-model-cache\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.751136 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.751042    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-home\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.751196 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.751177    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-tmp-dir\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.751254 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.751231    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-model-cache\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.753213 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.753181    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-dshm\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.753430 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.753414    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3070b8-12ff-4726-b3b2-499d758ba5fa-tls-certs\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.758136 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.758114    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8j7m\" (UniqueName: \"kubernetes.io/projected/de3070b8-12ff-4726-b3b2-499d758ba5fa-kube-api-access-k8j7m\") pod \"explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") " pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:21.937114 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:21.937031    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:22.210501 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:22.210471    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"]
Sep 09 12:37:22.211684 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:37:22.211659    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde3070b8_12ff_4726_b3b2_499d758ba5fa.slice/crio-71406877caae3257c6b205e4033b60774d9d28ebb725e5f81947505dcf801f94 WatchSource:0}: Error finding container 71406877caae3257c6b205e4033b60774d9d28ebb725e5f81947505dcf801f94: Status 404 returned error can't find the container with id 71406877caae3257c6b205e4033b60774d9d28ebb725e5f81947505dcf801f94
Sep 09 12:37:22.662603 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:22.662558    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj" event={"ID":"de3070b8-12ff-4726-b3b2-499d758ba5fa","Type":"ContainerStarted","Data":"3b4410e9330cf35049d9b2f4d8960253e2e7fb30dfa4c345a57131b900e7b276"}
Sep 09 12:37:22.662603 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:22.662609    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj" event={"ID":"de3070b8-12ff-4726-b3b2-499d758ba5fa","Type":"ContainerStarted","Data":"71406877caae3257c6b205e4033b60774d9d28ebb725e5f81947505dcf801f94"}
Sep 09 12:37:22.681943 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:22.681883    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj" podStartSLOduration=1.681865948 podStartE2EDuration="1.681865948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:37:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:37:22.678219239 +0000 UTC m=+1062.374784391" watchObservedRunningTime="2026-09-09 12:37:22.681865948 +0000 UTC m=+1062.378431091"
Sep 09 12:37:30.100185 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.100148    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"]
Sep 09 12:37:30.104181 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.104159    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.106121 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.106098    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2475a5c7-openshift-default-dockercfg-plp95\""
Sep 09 12:37:30.116630 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.116603    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"]
Sep 09 12:37:30.236660 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.236622    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7l6d\" (UniqueName: \"kubernetes.io/projected/e657458c-393c-40e9-9c67-256e0855d62a-kube-api-access-p7l6d\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.236660 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.236662    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-istio-data\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.236977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.236746    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-workload-socket\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.236977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.236843    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-credential-socket\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.236977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.236940    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-workload-certs\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.236977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.236973    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e657458c-393c-40e9-9c67-256e0855d62a-istio-podinfo\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.237167 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.236992    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e657458c-393c-40e9-9c67-256e0855d62a-istio-token\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.237167 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.237009    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-istio-envoy\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.237167 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.237037    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e657458c-393c-40e9-9c67-256e0855d62a-istiod-ca-cert\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.337709 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.337669    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-credential-socket\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.337918 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.337746    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-workload-certs\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.337918 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.337782    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e657458c-393c-40e9-9c67-256e0855d62a-istio-podinfo\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.337918 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.337815    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e657458c-393c-40e9-9c67-256e0855d62a-istio-token\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.337918 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.337841    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-istio-envoy\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.338166 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.337993    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e657458c-393c-40e9-9c67-256e0855d62a-istiod-ca-cert\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.338166 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.338091    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7l6d\" (UniqueName: \"kubernetes.io/projected/e657458c-393c-40e9-9c67-256e0855d62a-kube-api-access-p7l6d\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.338166 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.338122    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-istio-data\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.338321 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.338194    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-credential-socket\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.338321 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.338218    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-workload-certs\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.338321 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.338195    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-workload-socket\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.338509 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.338480    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-istio-data\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.338576 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.338510    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-workload-socket\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.338701 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.338675    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e657458c-393c-40e9-9c67-256e0855d62a-istiod-ca-cert\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.340360 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.340337    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-istio-envoy\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.340908 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.340869    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e657458c-393c-40e9-9c67-256e0855d62a-istio-podinfo\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.345678 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.345652    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e657458c-393c-40e9-9c67-256e0855d62a-istio-token\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.346136 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.346115    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7l6d\" (UniqueName: \"kubernetes.io/projected/e657458c-393c-40e9-9c67-256e0855d62a-kube-api-access-p7l6d\") pod \"e2e-mt-2475a5c7-openshift-default-765685846d-49tkw\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") " pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.416232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.416136    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:30.696311 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.696281    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"]
Sep 09 12:37:30.697700 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:37:30.697657    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode657458c_393c_40e9_9c67_256e0855d62a.slice/crio-fed5d41f6966253157992729c1070e30bb3e79cc0c5b3e465f8aaf36606b6029 WatchSource:0}: Error finding container fed5d41f6966253157992729c1070e30bb3e79cc0c5b3e465f8aaf36606b6029: Status 404 returned error can't find the container with id fed5d41f6966253157992729c1070e30bb3e79cc0c5b3e465f8aaf36606b6029
Sep 09 12:37:30.700097 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.700064    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:37:30.700173 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.700133    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:37:30.700173 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.700161    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:37:30.718367 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:30.718336    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" event={"ID":"e657458c-393c-40e9-9c67-256e0855d62a","Type":"ContainerStarted","Data":"fed5d41f6966253157992729c1070e30bb3e79cc0c5b3e465f8aaf36606b6029"}
Sep 09 12:37:31.725860 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:31.725821    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" event={"ID":"e657458c-393c-40e9-9c67-256e0855d62a","Type":"ContainerStarted","Data":"ebbbd067d6be6af2d0714efa7bc78ffc0ef2f0cef22e7fbe01350ef5dcb394cb"}
Sep 09 12:37:31.748534 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:31.748486    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" podStartSLOduration=1.748472034 podStartE2EDuration="1.748472034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:37:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:37:31.744977409 +0000 UTC m=+1071.441542572" watchObservedRunningTime="2026-09-09 12:37:31.748472034 +0000 UTC m=+1071.445037194"
Sep 09 12:37:31.937492 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:31.937460    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:31.937656 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:31.937514    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:31.949592 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:31.949567    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:32.416484 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:32.416414    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:32.421339 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:32.421316    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:32.732483 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:32.732390    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:32.733419 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:32.733402    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:32.743415 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:32.743394    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:37:34.840614 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:34.840576    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"]
Sep 09 12:37:34.845317 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:34.845292    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:34.847584 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:34.847561    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2475a5c7-serving-cert\""
Sep 09 12:37:34.855143 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:34.855116    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"]
Sep 09 12:37:34.985748 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:34.985707    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe07515f-446e-4271-b2a6-964fce42a587-openshift-service-ca\") pod \"maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:34.985947 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:34.985765    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvtk8\" (UniqueName: \"kubernetes.io/projected/fe07515f-446e-4271-b2a6-964fce42a587-kube-api-access-jvtk8\") pod \"maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:34.985947 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:34.985885    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe07515f-446e-4271-b2a6-964fce42a587-maas-api-tls\") pod \"maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:34.986047 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:34.985948    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe07515f-446e-4271-b2a6-964fce42a587-metrics-certs\") pod \"maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:35.087035 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:35.086998    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe07515f-446e-4271-b2a6-964fce42a587-openshift-service-ca\") pod \"maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:35.087227 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:35.087055    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvtk8\" (UniqueName: \"kubernetes.io/projected/fe07515f-446e-4271-b2a6-964fce42a587-kube-api-access-jvtk8\") pod \"maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:35.087227 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:35.087100    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe07515f-446e-4271-b2a6-964fce42a587-maas-api-tls\") pod \"maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:35.087227 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:35.087138    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe07515f-446e-4271-b2a6-964fce42a587-metrics-certs\") pod \"maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:35.087864 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:35.087819    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe07515f-446e-4271-b2a6-964fce42a587-openshift-service-ca\") pod \"maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:35.089791 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:35.089768    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe07515f-446e-4271-b2a6-964fce42a587-maas-api-tls\") pod \"maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:35.089944 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:35.089820    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe07515f-446e-4271-b2a6-964fce42a587-metrics-certs\") pod \"maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:35.095459 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:35.095398    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvtk8\" (UniqueName: \"kubernetes.io/projected/fe07515f-446e-4271-b2a6-964fce42a587-kube-api-access-jvtk8\") pod \"maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:35.158438 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:35.158405    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:35.440989 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:35.440955    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"]
Sep 09 12:37:35.442653 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:37:35.442627    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe07515f_446e_4271_b2a6_964fce42a587.slice/crio-6d9a760559c606d01e6e3a98133673476dafd360fe8eedb941dcce061a030b3d WatchSource:0}: Error finding container 6d9a760559c606d01e6e3a98133673476dafd360fe8eedb941dcce061a030b3d: Status 404 returned error can't find the container with id 6d9a760559c606d01e6e3a98133673476dafd360fe8eedb941dcce061a030b3d
Sep 09 12:37:35.445182 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:35.445162    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:37:35.752530 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:35.752492    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx" event={"ID":"fe07515f-446e-4271-b2a6-964fce42a587","Type":"ContainerStarted","Data":"6d9a760559c606d01e6e3a98133673476dafd360fe8eedb941dcce061a030b3d"}
Sep 09 12:37:36.760173 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:36.760140    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx" event={"ID":"fe07515f-446e-4271-b2a6-964fce42a587","Type":"ContainerStarted","Data":"68a95b13be6166c0eb45ed67d770260e82a24e91416915bef43bdaf3ce696676"}
Sep 09 12:37:36.760598 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:36.760237    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:36.780152 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:36.780106    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx" podStartSLOduration=2.403727376 podStartE2EDuration="2.780089176s" totalImagesPullingTime="376.3618ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:37:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:37:36.778097682 +0000 UTC m=+1076.474662851" watchObservedRunningTime="2026-09-09 12:37:36.780089176 +0000 UTC m=+1076.476654378"
Sep 09 12:37:42.782163 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:42.782124    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:37:48.774037 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:48.773998    2582 patch_prober.go:28] interesting pod/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 12:37:48.774467 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:48.774113    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" podUID="e657458c-393c-40e9-9c67-256e0855d62a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 12:37:50.469184 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:50.469081    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"]
Sep 09 12:37:50.858255 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:50.858225    2582 generic.go:356] "Generic (PLEG): container finished" podID="e657458c-393c-40e9-9c67-256e0855d62a" containerID="ebbbd067d6be6af2d0714efa7bc78ffc0ef2f0cef22e7fbe01350ef5dcb394cb" exitCode=0
Sep 09 12:37:50.858255 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:50.858259    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" event={"ID":"e657458c-393c-40e9-9c67-256e0855d62a","Type":"ContainerDied","Data":"ebbbd067d6be6af2d0714efa7bc78ffc0ef2f0cef22e7fbe01350ef5dcb394cb"}
Sep 09 12:37:50.858630 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:50.858615    2582 scope.go:117] "RemoveContainer" containerID="ebbbd067d6be6af2d0714efa7bc78ffc0ef2f0cef22e7fbe01350ef5dcb394cb"
Sep 09 12:37:50.859694 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:50.859658    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:37:50.859815 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:50.859751    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:37:50.859815 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:50.859802    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:37:51.874128 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:51.874085    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" event={"ID":"e657458c-393c-40e9-9c67-256e0855d62a","Type":"ContainerStarted","Data":"2acd4162a8aa8eeb3159b83a203debd148bb0f5241d2f688c9f80d8ccf589a39"}
Sep 09 12:37:52.417312 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:52.417267    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:52.417636 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:52.417531    2582 patch_prober.go:28] interesting pod/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 09 12:37:52.417636 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:52.417613    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" podUID="e657458c-393c-40e9-9c67-256e0855d62a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 09 12:37:53.417296 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:53.417258    2582 patch_prober.go:28] interesting pod/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 09 12:37:53.417692 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:53.417331    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" podUID="e657458c-393c-40e9-9c67-256e0855d62a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 09 12:37:54.416922 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:54.416878    2582 patch_prober.go:28] interesting pod/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 09 12:37:54.417108 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:54.416954    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" podUID="e657458c-393c-40e9-9c67-256e0855d62a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 09 12:37:55.416571 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:55.416535    2582 patch_prober.go:28] interesting pod/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 09 12:37:55.416998 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:55.416594    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" podUID="e657458c-393c-40e9-9c67-256e0855d62a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 09 12:37:55.743802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:55.743769    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"]
Sep 09 12:37:56.416669 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:56.416632    2582 patch_prober.go:28] interesting pod/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 09 12:37:56.417225 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:56.416701    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" podUID="e657458c-393c-40e9-9c67-256e0855d62a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 09 12:37:57.416842 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:57.416800    2582 patch_prober.go:28] interesting pod/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 09 12:37:57.417243 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:57.416896    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" podUID="e657458c-393c-40e9-9c67-256e0855d62a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 09 12:37:58.419871 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:58.419821    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:58.420344 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:58.420328    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:37:58.420962 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:37:58.420942    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:38:05.491403 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.491362    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj" podUID="de3070b8-12ff-4726-b3b2-499d758ba5fa" containerName="main" containerID="cri-o://3b4410e9330cf35049d9b2f4d8960253e2e7fb30dfa4c345a57131b900e7b276" gracePeriod=45
Sep 09 12:38:05.755557 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.755535    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:38:05.815457 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.815428    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3070b8-12ff-4726-b3b2-499d758ba5fa-tls-certs\") pod \"de3070b8-12ff-4726-b3b2-499d758ba5fa\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") "
Sep 09 12:38:05.815583 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.815540    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-tmp-dir\") pod \"de3070b8-12ff-4726-b3b2-499d758ba5fa\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") "
Sep 09 12:38:05.815635 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.815629    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-model-cache\") pod \"de3070b8-12ff-4726-b3b2-499d758ba5fa\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") "
Sep 09 12:38:05.815676 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.815651    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-home\") pod \"de3070b8-12ff-4726-b3b2-499d758ba5fa\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") "
Sep 09 12:38:05.815676 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.815667    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8j7m\" (UniqueName: \"kubernetes.io/projected/de3070b8-12ff-4726-b3b2-499d758ba5fa-kube-api-access-k8j7m\") pod \"de3070b8-12ff-4726-b3b2-499d758ba5fa\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") "
Sep 09 12:38:05.815746 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.815690    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-dshm\") pod \"de3070b8-12ff-4726-b3b2-499d758ba5fa\" (UID: \"de3070b8-12ff-4726-b3b2-499d758ba5fa\") "
Sep 09 12:38:05.815832 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.815807    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "de3070b8-12ff-4726-b3b2-499d758ba5fa" (UID: "de3070b8-12ff-4726-b3b2-499d758ba5fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:05.815919 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.815900    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "de3070b8-12ff-4726-b3b2-499d758ba5fa" (UID: "de3070b8-12ff-4726-b3b2-499d758ba5fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:05.815987 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.815915    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-home" (OuterVolumeSpecName: "home") pod "de3070b8-12ff-4726-b3b2-499d758ba5fa" (UID: "de3070b8-12ff-4726-b3b2-499d758ba5fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:05.816157 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.816106    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-model-cache\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:05.816157 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.816122    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-home\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:05.816157 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.816147    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-tmp-dir\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:05.817723 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.817698    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de3070b8-12ff-4726-b3b2-499d758ba5fa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "de3070b8-12ff-4726-b3b2-499d758ba5fa" (UID: "de3070b8-12ff-4726-b3b2-499d758ba5fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:38:05.817919 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.817893    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-dshm" (OuterVolumeSpecName: "dshm") pod "de3070b8-12ff-4726-b3b2-499d758ba5fa" (UID: "de3070b8-12ff-4726-b3b2-499d758ba5fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:05.817983 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.817923    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de3070b8-12ff-4726-b3b2-499d758ba5fa-kube-api-access-k8j7m" (OuterVolumeSpecName: "kube-api-access-k8j7m") pod "de3070b8-12ff-4726-b3b2-499d758ba5fa" (UID: "de3070b8-12ff-4726-b3b2-499d758ba5fa"). InnerVolumeSpecName "kube-api-access-k8j7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:05.916587 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.916564    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3070b8-12ff-4726-b3b2-499d758ba5fa-dshm\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:05.916587 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.916582    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3070b8-12ff-4726-b3b2-499d758ba5fa-tls-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:05.916724 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.916593    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8j7m\" (UniqueName: \"kubernetes.io/projected/de3070b8-12ff-4726-b3b2-499d758ba5fa-kube-api-access-k8j7m\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:05.982441 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.982403    2582 generic.go:356] "Generic (PLEG): container finished" podID="de3070b8-12ff-4726-b3b2-499d758ba5fa" containerID="3b4410e9330cf35049d9b2f4d8960253e2e7fb30dfa4c345a57131b900e7b276" exitCode=0
Sep 09 12:38:05.982568 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.982450    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"
Sep 09 12:38:05.982568 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.982481    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj" event={"ID":"de3070b8-12ff-4726-b3b2-499d758ba5fa","Type":"ContainerDied","Data":"3b4410e9330cf35049d9b2f4d8960253e2e7fb30dfa4c345a57131b900e7b276"}
Sep 09 12:38:05.982568 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.982520    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj" event={"ID":"de3070b8-12ff-4726-b3b2-499d758ba5fa","Type":"ContainerDied","Data":"71406877caae3257c6b205e4033b60774d9d28ebb725e5f81947505dcf801f94"}
Sep 09 12:38:05.982568 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.982542    2582 scope.go:117] "RemoveContainer" containerID="3b4410e9330cf35049d9b2f4d8960253e2e7fb30dfa4c345a57131b900e7b276"
Sep 09 12:38:05.993701 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.993681    2582 scope.go:117] "RemoveContainer" containerID="3b4410e9330cf35049d9b2f4d8960253e2e7fb30dfa4c345a57131b900e7b276"
Sep 09 12:38:05.994096 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:38:05.994072    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b4410e9330cf35049d9b2f4d8960253e2e7fb30dfa4c345a57131b900e7b276\": container with ID starting with 3b4410e9330cf35049d9b2f4d8960253e2e7fb30dfa4c345a57131b900e7b276 not found: ID does not exist" containerID="3b4410e9330cf35049d9b2f4d8960253e2e7fb30dfa4c345a57131b900e7b276"
Sep 09 12:38:05.994205 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:05.994107    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b4410e9330cf35049d9b2f4d8960253e2e7fb30dfa4c345a57131b900e7b276"} err="failed to get container status \"3b4410e9330cf35049d9b2f4d8960253e2e7fb30dfa4c345a57131b900e7b276\": rpc error: code = NotFound desc = could not find container \"3b4410e9330cf35049d9b2f4d8960253e2e7fb30dfa4c345a57131b900e7b276\": container with ID starting with 3b4410e9330cf35049d9b2f4d8960253e2e7fb30dfa4c345a57131b900e7b276 not found: ID does not exist"
Sep 09 12:38:06.004532 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:06.004508    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"]
Sep 09 12:38:06.013352 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:06.013303    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-0f93fa/explicit-model-0f93fa-kserve-58cdd8dc79-x7dwj"]
Sep 09 12:38:06.826092 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:06.826057    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de3070b8-12ff-4726-b3b2-499d758ba5fa" path="/var/lib/kubelet/pods/de3070b8-12ff-4726-b3b2-499d758ba5fa/volumes"
Sep 09 12:38:07.673978 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:07.673949    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"]
Sep 09 12:38:07.674206 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:07.674184    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h" podUID="38f88501-a343-42df-870c-c6fa0bfcbf25" containerName="maas-api" containerID="cri-o://78ed20d59d38508b9976533223bdabdc42bef69985db00cd55a23d5679c1f812" gracePeriod=30
Sep 09 12:38:07.978943 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:07.978903    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h" podUID="38f88501-a343-42df-870c-c6fa0bfcbf25" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.61:8443/health\": dial tcp 10.133.0.61:8443: connect: connection refused"
Sep 09 12:38:08.001887 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:08.001819    2582 generic.go:356] "Generic (PLEG): container finished" podID="38f88501-a343-42df-870c-c6fa0bfcbf25" containerID="78ed20d59d38508b9976533223bdabdc42bef69985db00cd55a23d5679c1f812" exitCode=0
Sep 09 12:38:08.002026 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:08.001910    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h" event={"ID":"38f88501-a343-42df-870c-c6fa0bfcbf25","Type":"ContainerDied","Data":"78ed20d59d38508b9976533223bdabdc42bef69985db00cd55a23d5679c1f812"}
Sep 09 12:38:08.157648 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:08.157626    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:38:08.244754 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:08.242046    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nd4z4\" (UniqueName: \"kubernetes.io/projected/38f88501-a343-42df-870c-c6fa0bfcbf25-kube-api-access-nd4z4\") pod \"38f88501-a343-42df-870c-c6fa0bfcbf25\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") "
Sep 09 12:38:08.244754 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:08.242119    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38f88501-a343-42df-870c-c6fa0bfcbf25-openshift-service-ca\") pod \"38f88501-a343-42df-870c-c6fa0bfcbf25\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") "
Sep 09 12:38:08.244754 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:08.242249    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38f88501-a343-42df-870c-c6fa0bfcbf25-maas-api-tls\") pod \"38f88501-a343-42df-870c-c6fa0bfcbf25\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") "
Sep 09 12:38:08.244754 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:08.242300    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38f88501-a343-42df-870c-c6fa0bfcbf25-metrics-certs\") pod \"38f88501-a343-42df-870c-c6fa0bfcbf25\" (UID: \"38f88501-a343-42df-870c-c6fa0bfcbf25\") "
Sep 09 12:38:08.246875 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:08.245572    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38f88501-a343-42df-870c-c6fa0bfcbf25-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "38f88501-a343-42df-870c-c6fa0bfcbf25" (UID: "38f88501-a343-42df-870c-c6fa0bfcbf25"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:38:08.246875 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:08.245931    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38f88501-a343-42df-870c-c6fa0bfcbf25-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "38f88501-a343-42df-870c-c6fa0bfcbf25" (UID: "38f88501-a343-42df-870c-c6fa0bfcbf25"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:08.250462 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:08.250430    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38f88501-a343-42df-870c-c6fa0bfcbf25-kube-api-access-nd4z4" (OuterVolumeSpecName: "kube-api-access-nd4z4") pod "38f88501-a343-42df-870c-c6fa0bfcbf25" (UID: "38f88501-a343-42df-870c-c6fa0bfcbf25"). InnerVolumeSpecName "kube-api-access-nd4z4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:08.250717 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:08.250693    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38f88501-a343-42df-870c-c6fa0bfcbf25-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "38f88501-a343-42df-870c-c6fa0bfcbf25" (UID: "38f88501-a343-42df-870c-c6fa0bfcbf25"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:38:08.343844 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:08.343814    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38f88501-a343-42df-870c-c6fa0bfcbf25-maas-api-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:08.343844 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:08.343843    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38f88501-a343-42df-870c-c6fa0bfcbf25-metrics-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:08.344053 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:08.343881    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd4z4\" (UniqueName: \"kubernetes.io/projected/38f88501-a343-42df-870c-c6fa0bfcbf25-kube-api-access-nd4z4\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:08.344053 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:08.343891    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38f88501-a343-42df-870c-c6fa0bfcbf25-openshift-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:09.009473 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:09.009445    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"
Sep 09 12:38:09.009892 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:09.009471    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h" event={"ID":"38f88501-a343-42df-870c-c6fa0bfcbf25","Type":"ContainerDied","Data":"c69e27b7d72af77d4508ccd5bcdec6c2df1e8dd155437d2e40bcdd8798564a1d"}
Sep 09 12:38:09.009892 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:09.009525    2582 scope.go:117] "RemoveContainer" containerID="78ed20d59d38508b9976533223bdabdc42bef69985db00cd55a23d5679c1f812"
Sep 09 12:38:09.029051 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:09.029026    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"]
Sep 09 12:38:09.031655 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:09.031631    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1e223f-fbb5c77f8-xtb4h"]
Sep 09 12:38:10.144193 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.144144    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"]
Sep 09 12:38:10.144658 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.144602    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38f88501-a343-42df-870c-c6fa0bfcbf25" containerName="maas-api"
Sep 09 12:38:10.144658 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.144615    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="38f88501-a343-42df-870c-c6fa0bfcbf25" containerName="maas-api"
Sep 09 12:38:10.144658 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.144623    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de3070b8-12ff-4726-b3b2-499d758ba5fa" containerName="main"
Sep 09 12:38:10.144658 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.144628    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="de3070b8-12ff-4726-b3b2-499d758ba5fa" containerName="main"
Sep 09 12:38:10.144930 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.144717    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="de3070b8-12ff-4726-b3b2-499d758ba5fa" containerName="main"
Sep 09 12:38:10.144930 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.144724    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="38f88501-a343-42df-870c-c6fa0bfcbf25" containerName="maas-api"
Sep 09 12:38:10.149186 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.149158    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.151529 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.151506    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-6a8ecc-openshift-default-dockercfg-rksnv\""
Sep 09 12:38:10.159452 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.159430    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"]
Sep 09 12:38:10.262192 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.262156    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwxkl\" (UniqueName: \"kubernetes.io/projected/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-kube-api-access-wwxkl\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.262192 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.262191    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-credential-socket\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.262414 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.262211    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-podinfo\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.262414 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.262294    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-envoy\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.262414 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.262361    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-data\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.262414 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.262404    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-workload-socket\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.262582 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.262423    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-workload-certs\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.262582 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.262476    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-token\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.262582 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.262498    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istiod-ca-cert\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.364026 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.363988    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-envoy\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.364207 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.364054    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-data\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.364207 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.364106    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-workload-socket\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.364207 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.364131    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-workload-certs\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.364207 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.364167    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-token\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.364397 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.364353    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istiod-ca-cert\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.364451 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.364428    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-data\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.364508 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.364454    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwxkl\" (UniqueName: \"kubernetes.io/projected/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-kube-api-access-wwxkl\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.364508 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.364499    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-credential-socket\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.364604 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.364510    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-workload-socket\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.364604 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.364533    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-podinfo\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.364604 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.364551    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-workload-certs\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.364793 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.364774    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-credential-socket\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.365073 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.365054    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istiod-ca-cert\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.366586 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.366561    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-envoy\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.367132 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.367115    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-podinfo\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.372372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.372350    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-token\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.372924 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.372905    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwxkl\" (UniqueName: \"kubernetes.io/projected/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-kube-api-access-wwxkl\") pod \"e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") " pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.461252 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.461200    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:10.742891 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.742864    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"]
Sep 09 12:38:10.745251 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:38:10.745226    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod115fa14f_c994_4ba6_bd9e_9c47507fb6b3.slice/crio-595571a61762aa623e12cc259f72b465b49a1f9abbf190f99624acf83e8a4e25 WatchSource:0}: Error finding container 595571a61762aa623e12cc259f72b465b49a1f9abbf190f99624acf83e8a4e25: Status 404 returned error can't find the container with id 595571a61762aa623e12cc259f72b465b49a1f9abbf190f99624acf83e8a4e25
Sep 09 12:38:10.747933 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.747899    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:38:10.748024 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.747969    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:38:10.748024 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.747997    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:38:10.766031 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.766003    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92" podUID="0c64328a-8bf0-4b59-bf33-b337780a0773" containerName="main" containerID="cri-o://1466ab38c2a37ebd966531d20b93d16ec0959b9ccf476a85e8043358d2a6489d" gracePeriod=45
Sep 09 12:38:10.825722 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:10.825692    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38f88501-a343-42df-870c-c6fa0bfcbf25" path="/var/lib/kubelet/pods/38f88501-a343-42df-870c-c6fa0bfcbf25/volumes"
Sep 09 12:38:11.017150 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.017122    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:38:11.023812 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.023786    2582 generic.go:356] "Generic (PLEG): container finished" podID="0c64328a-8bf0-4b59-bf33-b337780a0773" containerID="1466ab38c2a37ebd966531d20b93d16ec0959b9ccf476a85e8043358d2a6489d" exitCode=0
Sep 09 12:38:11.023955 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.023894    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92" event={"ID":"0c64328a-8bf0-4b59-bf33-b337780a0773","Type":"ContainerDied","Data":"1466ab38c2a37ebd966531d20b93d16ec0959b9ccf476a85e8043358d2a6489d"}
Sep 09 12:38:11.023955 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.023901    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"
Sep 09 12:38:11.023955 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.023944    2582 scope.go:117] "RemoveContainer" containerID="1466ab38c2a37ebd966531d20b93d16ec0959b9ccf476a85e8043358d2a6489d"
Sep 09 12:38:11.024118 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.023931    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92" event={"ID":"0c64328a-8bf0-4b59-bf33-b337780a0773","Type":"ContainerDied","Data":"3558ed97382d8ae6b17e8c802d55196dfd1769536defc4396448b09d226ea9d4"}
Sep 09 12:38:11.025701 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.025679    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz" event={"ID":"115fa14f-c994-4ba6-bd9e-9c47507fb6b3","Type":"ContainerStarted","Data":"f90d1dffcf3cce89b26fbd0476e2c7f9f031d86eab8a1377c897a3e6e98a264d"}
Sep 09 12:38:11.025808 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.025727    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz" event={"ID":"115fa14f-c994-4ba6-bd9e-9c47507fb6b3","Type":"ContainerStarted","Data":"595571a61762aa623e12cc259f72b465b49a1f9abbf190f99624acf83e8a4e25"}
Sep 09 12:38:11.034498 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.034474    2582 scope.go:117] "RemoveContainer" containerID="1466ab38c2a37ebd966531d20b93d16ec0959b9ccf476a85e8043358d2a6489d"
Sep 09 12:38:11.051003 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:38:11.050963    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1466ab38c2a37ebd966531d20b93d16ec0959b9ccf476a85e8043358d2a6489d\": container with ID starting with 1466ab38c2a37ebd966531d20b93d16ec0959b9ccf476a85e8043358d2a6489d not found: ID does not exist" containerID="1466ab38c2a37ebd966531d20b93d16ec0959b9ccf476a85e8043358d2a6489d"
Sep 09 12:38:11.051131 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.051015    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1466ab38c2a37ebd966531d20b93d16ec0959b9ccf476a85e8043358d2a6489d"} err="failed to get container status \"1466ab38c2a37ebd966531d20b93d16ec0959b9ccf476a85e8043358d2a6489d\": rpc error: code = NotFound desc = could not find container \"1466ab38c2a37ebd966531d20b93d16ec0959b9ccf476a85e8043358d2a6489d\": container with ID starting with 1466ab38c2a37ebd966531d20b93d16ec0959b9ccf476a85e8043358d2a6489d not found: ID does not exist"
Sep 09 12:38:11.057718 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.057675    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz" podStartSLOduration=1.057661921 podStartE2EDuration="1.057661921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:38:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:38:11.055523009 +0000 UTC m=+1110.752088162" watchObservedRunningTime="2026-09-09 12:38:11.057661921 +0000 UTC m=+1110.754227073"
Sep 09 12:38:11.172669 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.172616    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-home\") pod \"0c64328a-8bf0-4b59-bf33-b337780a0773\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") "
Sep 09 12:38:11.173222 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.172723    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-model-cache\") pod \"0c64328a-8bf0-4b59-bf33-b337780a0773\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") "
Sep 09 12:38:11.173222 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.172754    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-dshm\") pod \"0c64328a-8bf0-4b59-bf33-b337780a0773\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") "
Sep 09 12:38:11.173222 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.172797    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c64328a-8bf0-4b59-bf33-b337780a0773-tls-certs\") pod \"0c64328a-8bf0-4b59-bf33-b337780a0773\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") "
Sep 09 12:38:11.173222 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.172838    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-tmp-dir\") pod \"0c64328a-8bf0-4b59-bf33-b337780a0773\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") "
Sep 09 12:38:11.173222 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.172959    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkpfp\" (UniqueName: \"kubernetes.io/projected/0c64328a-8bf0-4b59-bf33-b337780a0773-kube-api-access-tkpfp\") pod \"0c64328a-8bf0-4b59-bf33-b337780a0773\" (UID: \"0c64328a-8bf0-4b59-bf33-b337780a0773\") "
Sep 09 12:38:11.174648 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.174613    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0c64328a-8bf0-4b59-bf33-b337780a0773" (UID: "0c64328a-8bf0-4b59-bf33-b337780a0773"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:11.176171 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.174940    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-home" (OuterVolumeSpecName: "home") pod "0c64328a-8bf0-4b59-bf33-b337780a0773" (UID: "0c64328a-8bf0-4b59-bf33-b337780a0773"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:11.176171 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.175054    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-model-cache" (OuterVolumeSpecName: "model-cache") pod "0c64328a-8bf0-4b59-bf33-b337780a0773" (UID: "0c64328a-8bf0-4b59-bf33-b337780a0773"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:11.179157 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.178251    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c64328a-8bf0-4b59-bf33-b337780a0773-kube-api-access-tkpfp" (OuterVolumeSpecName: "kube-api-access-tkpfp") pod "0c64328a-8bf0-4b59-bf33-b337780a0773" (UID: "0c64328a-8bf0-4b59-bf33-b337780a0773"). InnerVolumeSpecName "kube-api-access-tkpfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:11.179157 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.178915    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c64328a-8bf0-4b59-bf33-b337780a0773-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0c64328a-8bf0-4b59-bf33-b337780a0773" (UID: "0c64328a-8bf0-4b59-bf33-b337780a0773"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:38:11.179157 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.179064    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-dshm" (OuterVolumeSpecName: "dshm") pod "0c64328a-8bf0-4b59-bf33-b337780a0773" (UID: "0c64328a-8bf0-4b59-bf33-b337780a0773"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:11.274971 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.274874    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-dshm\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:11.274971 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.274904    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c64328a-8bf0-4b59-bf33-b337780a0773-tls-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:11.274971 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.274914    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-tmp-dir\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:11.274971 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.274923    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkpfp\" (UniqueName: \"kubernetes.io/projected/0c64328a-8bf0-4b59-bf33-b337780a0773-kube-api-access-tkpfp\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:11.274971 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.274932    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-home\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:11.274971 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.274941    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c64328a-8bf0-4b59-bf33-b337780a0773-model-cache\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:11.351589 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.351557    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"]
Sep 09 12:38:11.355740 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.355717    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-5e5923/test-model-5e5923-kserve-57f9f4bcb7-67b92"]
Sep 09 12:38:11.461647 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.461613    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:11.466559 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:11.466537    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:12.033259 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:12.033226    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:12.034118 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:12.034100    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:12.826052 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:12.826011    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c64328a-8bf0-4b59-bf33-b337780a0773" path="/var/lib/kubelet/pods/0c64328a-8bf0-4b59-bf33-b337780a0773/volumes"
Sep 09 12:38:14.989553 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:14.989517    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"]
Sep 09 12:38:14.989935 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:14.989780    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn" podUID="afcba06e-529f-471a-a7fb-9df792f78662" containerName="istio-proxy" containerID="cri-o://150f932af157012650071abede143c5a2b361367cd993ba301f161e02acb8e17" gracePeriod=30
Sep 09 12:38:16.695679 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:16.695643    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"]
Sep 09 12:38:16.696192 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:16.695876    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx" podUID="fe07515f-446e-4271-b2a6-964fce42a587" containerName="maas-api" containerID="cri-o://68a95b13be6166c0eb45ed67d770260e82a24e91416915bef43bdaf3ce696676" gracePeriod=30
Sep 09 12:38:16.952675 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:16.952616    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:38:17.064741 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.064708    2582 generic.go:356] "Generic (PLEG): container finished" podID="fe07515f-446e-4271-b2a6-964fce42a587" containerID="68a95b13be6166c0eb45ed67d770260e82a24e91416915bef43bdaf3ce696676" exitCode=0
Sep 09 12:38:17.064924 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.064756    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"
Sep 09 12:38:17.064924 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.064771    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx" event={"ID":"fe07515f-446e-4271-b2a6-964fce42a587","Type":"ContainerDied","Data":"68a95b13be6166c0eb45ed67d770260e82a24e91416915bef43bdaf3ce696676"}
Sep 09 12:38:17.064924 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.064811    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx" event={"ID":"fe07515f-446e-4271-b2a6-964fce42a587","Type":"ContainerDied","Data":"6d9a760559c606d01e6e3a98133673476dafd360fe8eedb941dcce061a030b3d"}
Sep 09 12:38:17.064924 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.064835    2582 scope.go:117] "RemoveContainer" containerID="68a95b13be6166c0eb45ed67d770260e82a24e91416915bef43bdaf3ce696676"
Sep 09 12:38:17.076025 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.076005    2582 scope.go:117] "RemoveContainer" containerID="68a95b13be6166c0eb45ed67d770260e82a24e91416915bef43bdaf3ce696676"
Sep 09 12:38:17.076369 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:38:17.076348    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68a95b13be6166c0eb45ed67d770260e82a24e91416915bef43bdaf3ce696676\": container with ID starting with 68a95b13be6166c0eb45ed67d770260e82a24e91416915bef43bdaf3ce696676 not found: ID does not exist" containerID="68a95b13be6166c0eb45ed67d770260e82a24e91416915bef43bdaf3ce696676"
Sep 09 12:38:17.076446 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.076381    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68a95b13be6166c0eb45ed67d770260e82a24e91416915bef43bdaf3ce696676"} err="failed to get container status \"68a95b13be6166c0eb45ed67d770260e82a24e91416915bef43bdaf3ce696676\": rpc error: code = NotFound desc = could not find container \"68a95b13be6166c0eb45ed67d770260e82a24e91416915bef43bdaf3ce696676\": container with ID starting with 68a95b13be6166c0eb45ed67d770260e82a24e91416915bef43bdaf3ce696676 not found: ID does not exist"
Sep 09 12:38:17.133762 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.133731    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe07515f-446e-4271-b2a6-964fce42a587-metrics-certs\") pod \"fe07515f-446e-4271-b2a6-964fce42a587\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") "
Sep 09 12:38:17.133958 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.133823    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe07515f-446e-4271-b2a6-964fce42a587-openshift-service-ca\") pod \"fe07515f-446e-4271-b2a6-964fce42a587\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") "
Sep 09 12:38:17.133958 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.133866    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe07515f-446e-4271-b2a6-964fce42a587-maas-api-tls\") pod \"fe07515f-446e-4271-b2a6-964fce42a587\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") "
Sep 09 12:38:17.134054 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.133959    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvtk8\" (UniqueName: \"kubernetes.io/projected/fe07515f-446e-4271-b2a6-964fce42a587-kube-api-access-jvtk8\") pod \"fe07515f-446e-4271-b2a6-964fce42a587\" (UID: \"fe07515f-446e-4271-b2a6-964fce42a587\") "
Sep 09 12:38:17.134271 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.134241    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe07515f-446e-4271-b2a6-964fce42a587-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fe07515f-446e-4271-b2a6-964fce42a587" (UID: "fe07515f-446e-4271-b2a6-964fce42a587"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:17.136206 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.136173    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe07515f-446e-4271-b2a6-964fce42a587-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fe07515f-446e-4271-b2a6-964fce42a587" (UID: "fe07515f-446e-4271-b2a6-964fce42a587"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:38:17.136310 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.136204    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe07515f-446e-4271-b2a6-964fce42a587-kube-api-access-jvtk8" (OuterVolumeSpecName: "kube-api-access-jvtk8") pod "fe07515f-446e-4271-b2a6-964fce42a587" (UID: "fe07515f-446e-4271-b2a6-964fce42a587"). InnerVolumeSpecName "kube-api-access-jvtk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:17.136310 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.136248    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe07515f-446e-4271-b2a6-964fce42a587-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fe07515f-446e-4271-b2a6-964fce42a587" (UID: "fe07515f-446e-4271-b2a6-964fce42a587"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:38:17.235332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.235270    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvtk8\" (UniqueName: \"kubernetes.io/projected/fe07515f-446e-4271-b2a6-964fce42a587-kube-api-access-jvtk8\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:17.235332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.235294    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe07515f-446e-4271-b2a6-964fce42a587-metrics-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:17.235332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.235303    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe07515f-446e-4271-b2a6-964fce42a587-openshift-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:17.235332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.235313    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe07515f-446e-4271-b2a6-964fce42a587-maas-api-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:17.387362 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.387329    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"]
Sep 09 12:38:17.391041 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:17.391019    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2475a5c7-549dff8759-m4qdx"]
Sep 09 12:38:18.826441 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:18.826406    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe07515f-446e-4271-b2a6-964fce42a587" path="/var/lib/kubelet/pods/fe07515f-446e-4271-b2a6-964fce42a587/volumes"
Sep 09 12:38:20.255841 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.255780    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:38:20.362860 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.362824    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9g4b\" (UniqueName: \"kubernetes.io/projected/afcba06e-529f-471a-a7fb-9df792f78662-kube-api-access-p9g4b\") pod \"afcba06e-529f-471a-a7fb-9df792f78662\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") "
Sep 09 12:38:20.362998 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.362878    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-workload-socket\") pod \"afcba06e-529f-471a-a7fb-9df792f78662\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") "
Sep 09 12:38:20.362998 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.362906    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-workload-certs\") pod \"afcba06e-529f-471a-a7fb-9df792f78662\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") "
Sep 09 12:38:20.362998 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.362959    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-istio-data\") pod \"afcba06e-529f-471a-a7fb-9df792f78662\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") "
Sep 09 12:38:20.362998 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.362981    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-istio-envoy\") pod \"afcba06e-529f-471a-a7fb-9df792f78662\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") "
Sep 09 12:38:20.362998 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.362996    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afcba06e-529f-471a-a7fb-9df792f78662-istio-token\") pod \"afcba06e-529f-471a-a7fb-9df792f78662\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") "
Sep 09 12:38:20.363268 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.363043    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-credential-socket\") pod \"afcba06e-529f-471a-a7fb-9df792f78662\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") "
Sep 09 12:38:20.363268 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.363065    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afcba06e-529f-471a-a7fb-9df792f78662-istiod-ca-cert\") pod \"afcba06e-529f-471a-a7fb-9df792f78662\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") "
Sep 09 12:38:20.363268 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.363135    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afcba06e-529f-471a-a7fb-9df792f78662-istio-podinfo\") pod \"afcba06e-529f-471a-a7fb-9df792f78662\" (UID: \"afcba06e-529f-471a-a7fb-9df792f78662\") "
Sep 09 12:38:20.363268 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.363202    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "afcba06e-529f-471a-a7fb-9df792f78662" (UID: "afcba06e-529f-471a-a7fb-9df792f78662"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:20.363268 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.363212    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "afcba06e-529f-471a-a7fb-9df792f78662" (UID: "afcba06e-529f-471a-a7fb-9df792f78662"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:20.363268 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.363227    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-istio-data" (OuterVolumeSpecName: "istio-data") pod "afcba06e-529f-471a-a7fb-9df792f78662" (UID: "afcba06e-529f-471a-a7fb-9df792f78662"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:20.363504 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.363360    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "afcba06e-529f-471a-a7fb-9df792f78662" (UID: "afcba06e-529f-471a-a7fb-9df792f78662"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:20.363504 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.363468    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afcba06e-529f-471a-a7fb-9df792f78662-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "afcba06e-529f-471a-a7fb-9df792f78662" (UID: "afcba06e-529f-471a-a7fb-9df792f78662"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:20.363628 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.363575    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-credential-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:20.363628 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.363595    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afcba06e-529f-471a-a7fb-9df792f78662-istiod-ca-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:20.363628 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.363610    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-workload-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:20.363741 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.363627    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-workload-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:20.363801 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.363718    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-istio-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:20.365275 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.365244    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/afcba06e-529f-471a-a7fb-9df792f78662-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "afcba06e-529f-471a-a7fb-9df792f78662" (UID: "afcba06e-529f-471a-a7fb-9df792f78662"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:38:20.365275 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.365253    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "afcba06e-529f-471a-a7fb-9df792f78662" (UID: "afcba06e-529f-471a-a7fb-9df792f78662"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:20.365833 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.365814    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afcba06e-529f-471a-a7fb-9df792f78662-istio-token" (OuterVolumeSpecName: "istio-token") pod "afcba06e-529f-471a-a7fb-9df792f78662" (UID: "afcba06e-529f-471a-a7fb-9df792f78662"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:20.365912 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.365830    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afcba06e-529f-471a-a7fb-9df792f78662-kube-api-access-p9g4b" (OuterVolumeSpecName: "kube-api-access-p9g4b") pod "afcba06e-529f-471a-a7fb-9df792f78662" (UID: "afcba06e-529f-471a-a7fb-9df792f78662"). InnerVolumeSpecName "kube-api-access-p9g4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:20.464505 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.464475    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afcba06e-529f-471a-a7fb-9df792f78662-istio-podinfo\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:20.464505 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.464500    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9g4b\" (UniqueName: \"kubernetes.io/projected/afcba06e-529f-471a-a7fb-9df792f78662-kube-api-access-p9g4b\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:20.464672 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.464515    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afcba06e-529f-471a-a7fb-9df792f78662-istio-envoy\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:20.464672 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:20.464527    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afcba06e-529f-471a-a7fb-9df792f78662-istio-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:21.092151 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:21.092048    2582 generic.go:356] "Generic (PLEG): container finished" podID="afcba06e-529f-471a-a7fb-9df792f78662" containerID="150f932af157012650071abede143c5a2b361367cd993ba301f161e02acb8e17" exitCode=0
Sep 09 12:38:21.092151 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:21.092128    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"
Sep 09 12:38:21.092372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:21.092151    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn" event={"ID":"afcba06e-529f-471a-a7fb-9df792f78662","Type":"ContainerDied","Data":"150f932af157012650071abede143c5a2b361367cd993ba301f161e02acb8e17"}
Sep 09 12:38:21.092372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:21.092247    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn" event={"ID":"afcba06e-529f-471a-a7fb-9df792f78662","Type":"ContainerDied","Data":"93d113d8da059e342a7936123910376c244266a7c6ed52c1e25d814ff4170e86"}
Sep 09 12:38:21.092372 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:21.092275    2582 scope.go:117] "RemoveContainer" containerID="150f932af157012650071abede143c5a2b361367cd993ba301f161e02acb8e17"
Sep 09 12:38:21.102190 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:21.102169    2582 scope.go:117] "RemoveContainer" containerID="bd44f442fadac3da25165cf740311b94cbeb35336826b153fca717cbac030d64"
Sep 09 12:38:21.110599 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:21.110577    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"]
Sep 09 12:38:21.112102 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:21.111910    2582 scope.go:117] "RemoveContainer" containerID="150f932af157012650071abede143c5a2b361367cd993ba301f161e02acb8e17"
Sep 09 12:38:21.112355 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:38:21.112325    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"150f932af157012650071abede143c5a2b361367cd993ba301f161e02acb8e17\": container with ID starting with 150f932af157012650071abede143c5a2b361367cd993ba301f161e02acb8e17 not found: ID does not exist" containerID="150f932af157012650071abede143c5a2b361367cd993ba301f161e02acb8e17"
Sep 09 12:38:21.112468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:21.112363    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"150f932af157012650071abede143c5a2b361367cd993ba301f161e02acb8e17"} err="failed to get container status \"150f932af157012650071abede143c5a2b361367cd993ba301f161e02acb8e17\": rpc error: code = NotFound desc = could not find container \"150f932af157012650071abede143c5a2b361367cd993ba301f161e02acb8e17\": container with ID starting with 150f932af157012650071abede143c5a2b361367cd993ba301f161e02acb8e17 not found: ID does not exist"
Sep 09 12:38:21.112468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:21.112387    2582 scope.go:117] "RemoveContainer" containerID="bd44f442fadac3da25165cf740311b94cbeb35336826b153fca717cbac030d64"
Sep 09 12:38:21.112661 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:38:21.112638    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd44f442fadac3da25165cf740311b94cbeb35336826b153fca717cbac030d64\": container with ID starting with bd44f442fadac3da25165cf740311b94cbeb35336826b153fca717cbac030d64 not found: ID does not exist" containerID="bd44f442fadac3da25165cf740311b94cbeb35336826b153fca717cbac030d64"
Sep 09 12:38:21.112721 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:21.112671    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd44f442fadac3da25165cf740311b94cbeb35336826b153fca717cbac030d64"} err="failed to get container status \"bd44f442fadac3da25165cf740311b94cbeb35336826b153fca717cbac030d64\": rpc error: code = NotFound desc = could not find container \"bd44f442fadac3da25165cf740311b94cbeb35336826b153fca717cbac030d64\": container with ID starting with bd44f442fadac3da25165cf740311b94cbeb35336826b153fca717cbac030d64 not found: ID does not exist"
Sep 09 12:38:21.114586 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:21.114565    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-1e223f-openshift-default-7b4bd9d788-5f6pn"]
Sep 09 12:38:22.825948 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:22.825917    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afcba06e-529f-471a-a7fb-9df792f78662" path="/var/lib/kubelet/pods/afcba06e-529f-471a-a7fb-9df792f78662/volumes"
Sep 09 12:38:24.476734 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:24.476700    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"]
Sep 09 12:38:24.477285 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:24.476960    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" podUID="e657458c-393c-40e9-9c67-256e0855d62a" containerName="istio-proxy" containerID="cri-o://2acd4162a8aa8eeb3159b83a203debd148bb0f5241d2f688c9f80d8ccf589a39" gracePeriod=30
Sep 09 12:38:28.422933 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:28.422889    2582 patch_prober.go:28] interesting pod/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 12:38:28.423302 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:28.422945    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" podUID="e657458c-393c-40e9-9c67-256e0855d62a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 12:38:29.731634 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.731612    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:38:29.752930 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.752904    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-workload-certs\") pod \"e657458c-393c-40e9-9c67-256e0855d62a\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") "
Sep 09 12:38:29.753067 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.752954    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7l6d\" (UniqueName: \"kubernetes.io/projected/e657458c-393c-40e9-9c67-256e0855d62a-kube-api-access-p7l6d\") pod \"e657458c-393c-40e9-9c67-256e0855d62a\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") "
Sep 09 12:38:29.753067 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.752982    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e657458c-393c-40e9-9c67-256e0855d62a-istio-token\") pod \"e657458c-393c-40e9-9c67-256e0855d62a\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") "
Sep 09 12:38:29.753067 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.753021    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-istio-envoy\") pod \"e657458c-393c-40e9-9c67-256e0855d62a\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") "
Sep 09 12:38:29.753067 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.753054    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e657458c-393c-40e9-9c67-256e0855d62a-istiod-ca-cert\") pod \"e657458c-393c-40e9-9c67-256e0855d62a\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") "
Sep 09 12:38:29.753292 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.753103    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-istio-data\") pod \"e657458c-393c-40e9-9c67-256e0855d62a\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") "
Sep 09 12:38:29.753292 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.753197    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-credential-socket\") pod \"e657458c-393c-40e9-9c67-256e0855d62a\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") "
Sep 09 12:38:29.753292 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.753202    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e657458c-393c-40e9-9c67-256e0855d62a" (UID: "e657458c-393c-40e9-9c67-256e0855d62a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:29.753292 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.753269    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e657458c-393c-40e9-9c67-256e0855d62a-istio-podinfo\") pod \"e657458c-393c-40e9-9c67-256e0855d62a\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") "
Sep 09 12:38:29.753464 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.753301    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-workload-socket\") pod \"e657458c-393c-40e9-9c67-256e0855d62a\" (UID: \"e657458c-393c-40e9-9c67-256e0855d62a\") "
Sep 09 12:38:29.753464 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.753435    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e657458c-393c-40e9-9c67-256e0855d62a" (UID: "e657458c-393c-40e9-9c67-256e0855d62a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:29.753993 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.753669    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e657458c-393c-40e9-9c67-256e0855d62a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e657458c-393c-40e9-9c67-256e0855d62a" (UID: "e657458c-393c-40e9-9c67-256e0855d62a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:29.753993 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.753729    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-credential-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:29.753993 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.753749    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-workload-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:29.754221 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.754057    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e657458c-393c-40e9-9c67-256e0855d62a" (UID: "e657458c-393c-40e9-9c67-256e0855d62a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:29.754221 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.754171    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-istio-data" (OuterVolumeSpecName: "istio-data") pod "e657458c-393c-40e9-9c67-256e0855d62a" (UID: "e657458c-393c-40e9-9c67-256e0855d62a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:29.756523 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.756485    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e657458c-393c-40e9-9c67-256e0855d62a-istio-token" (OuterVolumeSpecName: "istio-token") pod "e657458c-393c-40e9-9c67-256e0855d62a" (UID: "e657458c-393c-40e9-9c67-256e0855d62a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:29.756760 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.756731    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e657458c-393c-40e9-9c67-256e0855d62a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e657458c-393c-40e9-9c67-256e0855d62a" (UID: "e657458c-393c-40e9-9c67-256e0855d62a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:38:29.756876 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.756812    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e657458c-393c-40e9-9c67-256e0855d62a" (UID: "e657458c-393c-40e9-9c67-256e0855d62a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:29.756983 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.756958    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e657458c-393c-40e9-9c67-256e0855d62a-kube-api-access-p7l6d" (OuterVolumeSpecName: "kube-api-access-p7l6d") pod "e657458c-393c-40e9-9c67-256e0855d62a" (UID: "e657458c-393c-40e9-9c67-256e0855d62a"). InnerVolumeSpecName "kube-api-access-p7l6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:29.854928 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.854903    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-istio-envoy\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:29.854928 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.854924    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e657458c-393c-40e9-9c67-256e0855d62a-istiod-ca-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:29.854928 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.854933    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-istio-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:29.855126 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.854941    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e657458c-393c-40e9-9c67-256e0855d62a-istio-podinfo\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:29.855126 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.854949    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e657458c-393c-40e9-9c67-256e0855d62a-workload-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:29.855126 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.854960    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7l6d\" (UniqueName: \"kubernetes.io/projected/e657458c-393c-40e9-9c67-256e0855d62a-kube-api-access-p7l6d\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:29.855126 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:29.854968    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e657458c-393c-40e9-9c67-256e0855d62a-istio-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:30.148169 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:30.148091    2582 generic.go:356] "Generic (PLEG): container finished" podID="e657458c-393c-40e9-9c67-256e0855d62a" containerID="2acd4162a8aa8eeb3159b83a203debd148bb0f5241d2f688c9f80d8ccf589a39" exitCode=0
Sep 09 12:38:30.148169 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:30.148141    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"
Sep 09 12:38:30.148322 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:30.148181    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" event={"ID":"e657458c-393c-40e9-9c67-256e0855d62a","Type":"ContainerDied","Data":"2acd4162a8aa8eeb3159b83a203debd148bb0f5241d2f688c9f80d8ccf589a39"}
Sep 09 12:38:30.148322 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:30.148240    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw" event={"ID":"e657458c-393c-40e9-9c67-256e0855d62a","Type":"ContainerDied","Data":"fed5d41f6966253157992729c1070e30bb3e79cc0c5b3e465f8aaf36606b6029"}
Sep 09 12:38:30.148322 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:30.148270    2582 scope.go:117] "RemoveContainer" containerID="2acd4162a8aa8eeb3159b83a203debd148bb0f5241d2f688c9f80d8ccf589a39"
Sep 09 12:38:30.158456 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:30.158441    2582 scope.go:117] "RemoveContainer" containerID="ebbbd067d6be6af2d0714efa7bc78ffc0ef2f0cef22e7fbe01350ef5dcb394cb"
Sep 09 12:38:30.166794 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:30.166778    2582 scope.go:117] "RemoveContainer" containerID="2acd4162a8aa8eeb3159b83a203debd148bb0f5241d2f688c9f80d8ccf589a39"
Sep 09 12:38:30.167108 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:38:30.167083    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2acd4162a8aa8eeb3159b83a203debd148bb0f5241d2f688c9f80d8ccf589a39\": container with ID starting with 2acd4162a8aa8eeb3159b83a203debd148bb0f5241d2f688c9f80d8ccf589a39 not found: ID does not exist" containerID="2acd4162a8aa8eeb3159b83a203debd148bb0f5241d2f688c9f80d8ccf589a39"
Sep 09 12:38:30.167190 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:30.167119    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2acd4162a8aa8eeb3159b83a203debd148bb0f5241d2f688c9f80d8ccf589a39"} err="failed to get container status \"2acd4162a8aa8eeb3159b83a203debd148bb0f5241d2f688c9f80d8ccf589a39\": rpc error: code = NotFound desc = could not find container \"2acd4162a8aa8eeb3159b83a203debd148bb0f5241d2f688c9f80d8ccf589a39\": container with ID starting with 2acd4162a8aa8eeb3159b83a203debd148bb0f5241d2f688c9f80d8ccf589a39 not found: ID does not exist"
Sep 09 12:38:30.167190 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:30.167141    2582 scope.go:117] "RemoveContainer" containerID="ebbbd067d6be6af2d0714efa7bc78ffc0ef2f0cef22e7fbe01350ef5dcb394cb"
Sep 09 12:38:30.167426 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:38:30.167409    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebbbd067d6be6af2d0714efa7bc78ffc0ef2f0cef22e7fbe01350ef5dcb394cb\": container with ID starting with ebbbd067d6be6af2d0714efa7bc78ffc0ef2f0cef22e7fbe01350ef5dcb394cb not found: ID does not exist" containerID="ebbbd067d6be6af2d0714efa7bc78ffc0ef2f0cef22e7fbe01350ef5dcb394cb"
Sep 09 12:38:30.167466 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:30.167432    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebbbd067d6be6af2d0714efa7bc78ffc0ef2f0cef22e7fbe01350ef5dcb394cb"} err="failed to get container status \"ebbbd067d6be6af2d0714efa7bc78ffc0ef2f0cef22e7fbe01350ef5dcb394cb\": rpc error: code = NotFound desc = could not find container \"ebbbd067d6be6af2d0714efa7bc78ffc0ef2f0cef22e7fbe01350ef5dcb394cb\": container with ID starting with ebbbd067d6be6af2d0714efa7bc78ffc0ef2f0cef22e7fbe01350ef5dcb394cb not found: ID does not exist"
Sep 09 12:38:30.171171 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:30.171151    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"]
Sep 09 12:38:30.174489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:30.174469    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2475a5c7-openshift-default-765685846d-49tkw"]
Sep 09 12:38:30.826563 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:30.826523    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e657458c-393c-40e9-9c67-256e0855d62a" path="/var/lib/kubelet/pods/e657458c-393c-40e9-9c67-256e0855d62a/volumes"
Sep 09 12:38:33.470689 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:33.470653    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"]
Sep 09 12:38:33.471178 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:33.470941    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb" podUID="f9f299b8-c998-444d-b2f1-3b47e15b4550" containerName="istio-proxy" containerID="cri-o://08b755407a35bed1245530d7b63a09ef2c7af18db3be8b50f18a7f41fea7fbe5" gracePeriod=30
Sep 09 12:38:38.733010 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.732962    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:38:38.838316 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.838291    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-data\") pod \"f9f299b8-c998-444d-b2f1-3b47e15b4550\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") "
Sep 09 12:38:38.838479 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.838332    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9f299b8-c998-444d-b2f1-3b47e15b4550-istiod-ca-cert\") pod \"f9f299b8-c998-444d-b2f1-3b47e15b4550\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") "
Sep 09 12:38:38.838479 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.838405    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-podinfo\") pod \"f9f299b8-c998-444d-b2f1-3b47e15b4550\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") "
Sep 09 12:38:38.838479 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.838448    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-workload-certs\") pod \"f9f299b8-c998-444d-b2f1-3b47e15b4550\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") "
Sep 09 12:38:38.838642 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.838488    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-envoy\") pod \"f9f299b8-c998-444d-b2f1-3b47e15b4550\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") "
Sep 09 12:38:38.838642 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.838513    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-credential-socket\") pod \"f9f299b8-c998-444d-b2f1-3b47e15b4550\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") "
Sep 09 12:38:38.838642 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.838565    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-token\") pod \"f9f299b8-c998-444d-b2f1-3b47e15b4550\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") "
Sep 09 12:38:38.838642 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.838578    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-data" (OuterVolumeSpecName: "istio-data") pod "f9f299b8-c998-444d-b2f1-3b47e15b4550" (UID: "f9f299b8-c998-444d-b2f1-3b47e15b4550"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:38.838642 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.838619    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8sck\" (UniqueName: \"kubernetes.io/projected/f9f299b8-c998-444d-b2f1-3b47e15b4550-kube-api-access-k8sck\") pod \"f9f299b8-c998-444d-b2f1-3b47e15b4550\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") "
Sep 09 12:38:38.838945 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.838651    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-workload-socket\") pod \"f9f299b8-c998-444d-b2f1-3b47e15b4550\" (UID: \"f9f299b8-c998-444d-b2f1-3b47e15b4550\") "
Sep 09 12:38:38.838945 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.838833    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f9f299b8-c998-444d-b2f1-3b47e15b4550" (UID: "f9f299b8-c998-444d-b2f1-3b47e15b4550"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:38.838945 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.838840    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f9f299b8-c998-444d-b2f1-3b47e15b4550" (UID: "f9f299b8-c998-444d-b2f1-3b47e15b4550"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:38.838945 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.838898    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9f299b8-c998-444d-b2f1-3b47e15b4550-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f9f299b8-c998-444d-b2f1-3b47e15b4550" (UID: "f9f299b8-c998-444d-b2f1-3b47e15b4550"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:38.839160 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.839088    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f9f299b8-c998-444d-b2f1-3b47e15b4550" (UID: "f9f299b8-c998-444d-b2f1-3b47e15b4550"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:38.839211 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.839171    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-workload-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:38.839211 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.839197    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-credential-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:38.839281 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.839214    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:38.839281 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.839230    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9f299b8-c998-444d-b2f1-3b47e15b4550-istiod-ca-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:38.841112 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.841089    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-token" (OuterVolumeSpecName: "istio-token") pod "f9f299b8-c998-444d-b2f1-3b47e15b4550" (UID: "f9f299b8-c998-444d-b2f1-3b47e15b4550"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:38.841238 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.841136    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9f299b8-c998-444d-b2f1-3b47e15b4550-kube-api-access-k8sck" (OuterVolumeSpecName: "kube-api-access-k8sck") pod "f9f299b8-c998-444d-b2f1-3b47e15b4550" (UID: "f9f299b8-c998-444d-b2f1-3b47e15b4550"). InnerVolumeSpecName "kube-api-access-k8sck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:38.841402 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.841380    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f9f299b8-c998-444d-b2f1-3b47e15b4550" (UID: "f9f299b8-c998-444d-b2f1-3b47e15b4550"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:38.841402 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.841391    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f9f299b8-c998-444d-b2f1-3b47e15b4550" (UID: "f9f299b8-c998-444d-b2f1-3b47e15b4550"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:38:38.939912 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.939880    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-podinfo\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:38.939912 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.939906    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-envoy\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:38.939912 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.939916    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9f299b8-c998-444d-b2f1-3b47e15b4550-istio-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:38.940120 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.939924    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8sck\" (UniqueName: \"kubernetes.io/projected/f9f299b8-c998-444d-b2f1-3b47e15b4550-kube-api-access-k8sck\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:38.940120 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.939933    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f299b8-c998-444d-b2f1-3b47e15b4550-workload-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.972217    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"]
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.972876    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e657458c-393c-40e9-9c67-256e0855d62a" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.972895    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e657458c-393c-40e9-9c67-256e0855d62a" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.972910    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e657458c-393c-40e9-9c67-256e0855d62a" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.972919    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e657458c-393c-40e9-9c67-256e0855d62a" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.972933    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afcba06e-529f-471a-a7fb-9df792f78662" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.972942    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="afcba06e-529f-471a-a7fb-9df792f78662" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.972950    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9f299b8-c998-444d-b2f1-3b47e15b4550" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.972958    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9f299b8-c998-444d-b2f1-3b47e15b4550" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.972969    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c64328a-8bf0-4b59-bf33-b337780a0773" containerName="main"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.972978    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c64328a-8bf0-4b59-bf33-b337780a0773" containerName="main"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.972995    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9f299b8-c998-444d-b2f1-3b47e15b4550" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.973004    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9f299b8-c998-444d-b2f1-3b47e15b4550" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.973015    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afcba06e-529f-471a-a7fb-9df792f78662" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.973022    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="afcba06e-529f-471a-a7fb-9df792f78662" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.973032    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe07515f-446e-4271-b2a6-964fce42a587" containerName="maas-api"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.973039    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe07515f-446e-4271-b2a6-964fce42a587" containerName="maas-api"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.973140    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9f299b8-c998-444d-b2f1-3b47e15b4550" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.973150    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9f299b8-c998-444d-b2f1-3b47e15b4550" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.973161    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e657458c-393c-40e9-9c67-256e0855d62a" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.973171    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="afcba06e-529f-471a-a7fb-9df792f78662" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.973184    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="afcba06e-529f-471a-a7fb-9df792f78662" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.973196    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c64328a-8bf0-4b59-bf33-b337780a0773" containerName="main"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.973204    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e657458c-393c-40e9-9c67-256e0855d62a" containerName="istio-proxy"
Sep 09 12:38:38.974802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.973214    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe07515f-446e-4271-b2a6-964fce42a587" containerName="maas-api"
Sep 09 12:38:38.991709 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.991638    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"]
Sep 09 12:38:38.991829 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.991761    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:38.994234 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:38.994200    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-283adb-serving-cert\""
Sep 09 12:38:39.040610 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.040583    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:39.040737 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.040625    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:39.040807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.040747    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r978\" (UniqueName: \"kubernetes.io/projected/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-kube-api-access-5r978\") pod \"maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:39.040807 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.040776    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:39.142174 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.142145    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:39.142313 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.142180    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:39.142396 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.142378    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r978\" (UniqueName: \"kubernetes.io/projected/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-kube-api-access-5r978\") pod \"maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:39.142466 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.142413    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:39.142771 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.142745    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:39.144718 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.144697    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:39.144817 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.144802    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:39.150301 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.150284    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r978\" (UniqueName: \"kubernetes.io/projected/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-kube-api-access-5r978\") pod \"maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:39.204163 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.204131    2582 generic.go:356] "Generic (PLEG): container finished" podID="f9f299b8-c998-444d-b2f1-3b47e15b4550" containerID="08b755407a35bed1245530d7b63a09ef2c7af18db3be8b50f18a7f41fea7fbe5" exitCode=0
Sep 09 12:38:39.204289 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.204185    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb" event={"ID":"f9f299b8-c998-444d-b2f1-3b47e15b4550","Type":"ContainerDied","Data":"08b755407a35bed1245530d7b63a09ef2c7af18db3be8b50f18a7f41fea7fbe5"}
Sep 09 12:38:39.204289 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.204241    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb" event={"ID":"f9f299b8-c998-444d-b2f1-3b47e15b4550","Type":"ContainerDied","Data":"71895cfc86f81ab8f22d1af245b53064e1cf52648cc0b595acf28df3a1596112"}
Sep 09 12:38:39.204289 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.204263    2582 scope.go:117] "RemoveContainer" containerID="08b755407a35bed1245530d7b63a09ef2c7af18db3be8b50f18a7f41fea7fbe5"
Sep 09 12:38:39.204289 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.204197    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"
Sep 09 12:38:39.213711 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.213696    2582 scope.go:117] "RemoveContainer" containerID="f820dfe10066e4e2a2f6ad261fa24fa62c12a408212c43fe125294966705ca17"
Sep 09 12:38:39.222030 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.222011    2582 scope.go:117] "RemoveContainer" containerID="08b755407a35bed1245530d7b63a09ef2c7af18db3be8b50f18a7f41fea7fbe5"
Sep 09 12:38:39.222287 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:38:39.222268    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08b755407a35bed1245530d7b63a09ef2c7af18db3be8b50f18a7f41fea7fbe5\": container with ID starting with 08b755407a35bed1245530d7b63a09ef2c7af18db3be8b50f18a7f41fea7fbe5 not found: ID does not exist" containerID="08b755407a35bed1245530d7b63a09ef2c7af18db3be8b50f18a7f41fea7fbe5"
Sep 09 12:38:39.222364 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.222294    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08b755407a35bed1245530d7b63a09ef2c7af18db3be8b50f18a7f41fea7fbe5"} err="failed to get container status \"08b755407a35bed1245530d7b63a09ef2c7af18db3be8b50f18a7f41fea7fbe5\": rpc error: code = NotFound desc = could not find container \"08b755407a35bed1245530d7b63a09ef2c7af18db3be8b50f18a7f41fea7fbe5\": container with ID starting with 08b755407a35bed1245530d7b63a09ef2c7af18db3be8b50f18a7f41fea7fbe5 not found: ID does not exist"
Sep 09 12:38:39.222364 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.222310    2582 scope.go:117] "RemoveContainer" containerID="f820dfe10066e4e2a2f6ad261fa24fa62c12a408212c43fe125294966705ca17"
Sep 09 12:38:39.222568 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:38:39.222551    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f820dfe10066e4e2a2f6ad261fa24fa62c12a408212c43fe125294966705ca17\": container with ID starting with f820dfe10066e4e2a2f6ad261fa24fa62c12a408212c43fe125294966705ca17 not found: ID does not exist" containerID="f820dfe10066e4e2a2f6ad261fa24fa62c12a408212c43fe125294966705ca17"
Sep 09 12:38:39.222635 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.222571    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f820dfe10066e4e2a2f6ad261fa24fa62c12a408212c43fe125294966705ca17"} err="failed to get container status \"f820dfe10066e4e2a2f6ad261fa24fa62c12a408212c43fe125294966705ca17\": rpc error: code = NotFound desc = could not find container \"f820dfe10066e4e2a2f6ad261fa24fa62c12a408212c43fe125294966705ca17\": container with ID starting with f820dfe10066e4e2a2f6ad261fa24fa62c12a408212c43fe125294966705ca17 not found: ID does not exist"
Sep 09 12:38:39.230117 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.230095    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"]
Sep 09 12:38:39.234450 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.234427    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-5e5923-openshift-default-574c9f5cf4-frltb"]
Sep 09 12:38:39.304742 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.304719    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:39.582598 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.582571    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"]
Sep 09 12:38:39.585120 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:38:39.585089    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20dd09ba_9e9f_4e97_a165_9f1a75ce4e83.slice/crio-fc00e7773ad439ab6039ffd2f2e145ae493c30856f6be628c42b1d84c9ec31c1 WatchSource:0}: Error finding container fc00e7773ad439ab6039ffd2f2e145ae493c30856f6be628c42b1d84c9ec31c1: Status 404 returned error can't find the container with id fc00e7773ad439ab6039ffd2f2e145ae493c30856f6be628c42b1d84c9ec31c1
Sep 09 12:38:39.968378 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.968350    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"]
Sep 09 12:38:39.972721 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.972693    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:39.976569 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.976508    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-a3f43f-openshift-default-dockercfg-9zn75\""
Sep 09 12:38:39.984022 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:39.983392    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"]
Sep 09 12:38:40.051993 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.051958    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-credential-socket\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.052106 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.051996    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-envoy\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.052106 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.052031    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-data\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.052211 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.052123    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-token\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.052211 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.052182    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-podinfo\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.052300 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.052259    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istiod-ca-cert\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.052344 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.052322    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt2q7\" (UniqueName: \"kubernetes.io/projected/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-kube-api-access-rt2q7\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.052407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.052390    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-workload-socket\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.052474 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.052449    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-workload-certs\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.153257 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.153215    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-podinfo\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.153447 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.153266    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istiod-ca-cert\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.153447 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.153305    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt2q7\" (UniqueName: \"kubernetes.io/projected/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-kube-api-access-rt2q7\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.153447 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.153351    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-workload-socket\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.153447 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.153400    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-workload-certs\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.153447 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.153433    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-credential-socket\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.153708 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.153458    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-envoy\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.153708 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.153488    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-data\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.153708 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.153529    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-token\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.153899 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.153822    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-workload-socket\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.154038 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.154002    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-credential-socket\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.154107 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.154036    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-workload-certs\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.154144 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.154103    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-data\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.154198 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.154171    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istiod-ca-cert\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.156085 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.156062    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-envoy\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.156264 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.156245    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-podinfo\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.162243 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.162222    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt2q7\" (UniqueName: \"kubernetes.io/projected/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-kube-api-access-rt2q7\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.162343 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.162271    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-token\") pod \"e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") " pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.213081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.212999    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n" event={"ID":"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83","Type":"ContainerStarted","Data":"54551dcbda3067f362ff48b5114ba0d04c08ffeec29888e07b0257fd65dca0bd"}
Sep 09 12:38:40.213081 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.213076    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n" event={"ID":"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83","Type":"ContainerStarted","Data":"fc00e7773ad439ab6039ffd2f2e145ae493c30856f6be628c42b1d84c9ec31c1"}
Sep 09 12:38:40.213287 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.213123    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:40.231073 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.231003    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n" podStartSLOduration=1.8573144510000001 podStartE2EDuration="2.230989001s" totalImagesPullingTime="373.67455ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:38:40.22900673 +0000 UTC m=+1139.925571895" watchObservedRunningTime="2026-09-09 12:38:40.230989001 +0000 UTC m=+1139.927554153"
Sep 09 12:38:40.308687 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.308654    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:40.588123 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.588099    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"]
Sep 09 12:38:40.590324 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:38:40.590296    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e03a3bd_f94b_4c2e_9df6_7c23fac9a2fe.slice/crio-f53f18e06cd0b9b3d078f1a05b620c2cc80cdff1ab9aa0ee92e8f8219b12f110 WatchSource:0}: Error finding container f53f18e06cd0b9b3d078f1a05b620c2cc80cdff1ab9aa0ee92e8f8219b12f110: Status 404 returned error can't find the container with id f53f18e06cd0b9b3d078f1a05b620c2cc80cdff1ab9aa0ee92e8f8219b12f110
Sep 09 12:38:40.592949 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.592908    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:38:40.593015 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.592998    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:38:40.593053 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.593041    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:38:40.827086 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:40.827053    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9f299b8-c998-444d-b2f1-3b47e15b4550" path="/var/lib/kubelet/pods/f9f299b8-c998-444d-b2f1-3b47e15b4550/volumes"
Sep 09 12:38:41.221642 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:41.221549    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" event={"ID":"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe","Type":"ContainerStarted","Data":"af5d54d439493cd4c132bbb6e3e861c86a1f10bb5f2d91d999e16a2ee5015a76"}
Sep 09 12:38:41.221642 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:41.221638    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" event={"ID":"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe","Type":"ContainerStarted","Data":"f53f18e06cd0b9b3d078f1a05b620c2cc80cdff1ab9aa0ee92e8f8219b12f110"}
Sep 09 12:38:41.243336 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:41.243287    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" podStartSLOduration=2.243273225 podStartE2EDuration="2.243273225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:38:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:38:41.241758705 +0000 UTC m=+1140.938323858" watchObservedRunningTime="2026-09-09 12:38:41.243273225 +0000 UTC m=+1140.939838376"
Sep 09 12:38:41.309304 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:41.309275    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:41.310939 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:41.310913    2582 patch_prober.go:28] interesting pod/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 09 12:38:41.311036 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:41.310970    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 09 12:38:42.313472 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:42.313445    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:43.056740 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:43.056703    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"]
Sep 09 12:38:43.057012 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:43.056984    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz" podUID="115fa14f-c994-4ba6-bd9e-9c47507fb6b3" containerName="istio-proxy" containerID="cri-o://f90d1dffcf3cce89b26fbd0476e2c7f9f031d86eab8a1377c897a3e6e98a264d" gracePeriod=30
Sep 09 12:38:43.233603 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:43.233511    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:43.234604 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:43.234571    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:38:44.440735 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:44.440042    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"]
Sep 09 12:38:44.445349 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:44.445330    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:44.447188 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:44.447169    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-a3f43f-serving-cert\""
Sep 09 12:38:44.453121 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:44.453102    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"]
Sep 09 12:38:44.601490 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:44.601458    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:44.601632 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:44.601499    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:44.601632 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:44.601525    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:44.601721 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:44.601634    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5nxs\" (UniqueName: \"kubernetes.io/projected/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-kube-api-access-r5nxs\") pod \"maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:44.702709 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:44.702623    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5nxs\" (UniqueName: \"kubernetes.io/projected/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-kube-api-access-r5nxs\") pod \"maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:44.702895 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:44.702741    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:44.702895 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:44.702776    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:44.702895 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:44.702809    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:44.703023 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:38:44.702904    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-serving-cert: secret "maas-api-e2e-shared-a-w2-a3f43f-serving-cert" not found
Sep 09 12:38:44.703023 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:38:44.702974    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-maas-api-tls podName:b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38 nodeName:}" failed. No retries permitted until 2026-09-09 12:38:45.202950552 +0000 UTC m=+1144.899515685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-maas-api-tls") pod "maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz" (UID: "b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38") : secret "maas-api-e2e-shared-a-w2-a3f43f-serving-cert" not found
Sep 09 12:38:44.703454 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:44.703433    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:44.705317 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:44.705301    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:44.712180 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:44.712158    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5nxs\" (UniqueName: \"kubernetes.io/projected/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-kube-api-access-r5nxs\") pod \"maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:45.207994 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:45.207941    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:45.210705 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:45.210680    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:45.360073 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:45.360043    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:45.634423 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:45.634398    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"]
Sep 09 12:38:45.637128 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:38:45.637099    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb97c50de_b01f_4b69_a1ba_ffd4a8ce5a38.slice/crio-e57adddba6707e4726752841b557bb5da43d96847fbf76e3e4e67cf0ffcb8493 WatchSource:0}: Error finding container e57adddba6707e4726752841b557bb5da43d96847fbf76e3e4e67cf0ffcb8493: Status 404 returned error can't find the container with id e57adddba6707e4726752841b557bb5da43d96847fbf76e3e4e67cf0ffcb8493
Sep 09 12:38:46.227393 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:46.227363    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:38:46.254722 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:46.254687    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz" event={"ID":"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38","Type":"ContainerStarted","Data":"61165a6f11aefdd64a740cf673b7e133159feca85a111e01b341bec8b8987af8"}
Sep 09 12:38:46.254932 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:46.254761    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz" event={"ID":"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38","Type":"ContainerStarted","Data":"e57adddba6707e4726752841b557bb5da43d96847fbf76e3e4e67cf0ffcb8493"}
Sep 09 12:38:46.254932 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:46.254808    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:46.273558 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:46.273499    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz" podStartSLOduration=1.9280477459999998 podStartE2EDuration="2.273485038s" totalImagesPullingTime="345.437292ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:38:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:38:46.271963209 +0000 UTC m=+1145.968528362" watchObservedRunningTime="2026-09-09 12:38:46.273485038 +0000 UTC m=+1145.970050189"
Sep 09 12:38:48.268413 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.268381    2582 generic.go:356] "Generic (PLEG): container finished" podID="115fa14f-c994-4ba6-bd9e-9c47507fb6b3" containerID="f90d1dffcf3cce89b26fbd0476e2c7f9f031d86eab8a1377c897a3e6e98a264d" exitCode=0
Sep 09 12:38:48.268808 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.268458    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz" event={"ID":"115fa14f-c994-4ba6-bd9e-9c47507fb6b3","Type":"ContainerDied","Data":"f90d1dffcf3cce89b26fbd0476e2c7f9f031d86eab8a1377c897a3e6e98a264d"}
Sep 09 12:38:48.323327 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.323262    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:48.439077 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.439041    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istiod-ca-cert\") pod \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") "
Sep 09 12:38:48.439077 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.439081    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-token\") pod \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") "
Sep 09 12:38:48.439301 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.439134    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwxkl\" (UniqueName: \"kubernetes.io/projected/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-kube-api-access-wwxkl\") pod \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") "
Sep 09 12:38:48.439301 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.439236    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-workload-certs\") pod \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") "
Sep 09 12:38:48.439301 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.439295    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-podinfo\") pod \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") "
Sep 09 12:38:48.439473 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.439374    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "115fa14f-c994-4ba6-bd9e-9c47507fb6b3" (UID: "115fa14f-c994-4ba6-bd9e-9c47507fb6b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:48.439473 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.439390    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-envoy\") pod \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") "
Sep 09 12:38:48.439473 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.439429    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-workload-socket\") pod \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") "
Sep 09 12:38:48.439473 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.439470    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-data\") pod \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") "
Sep 09 12:38:48.439667 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.439499    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-credential-socket\") pod \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\" (UID: \"115fa14f-c994-4ba6-bd9e-9c47507fb6b3\") "
Sep 09 12:38:48.439667 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.439557    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "115fa14f-c994-4ba6-bd9e-9c47507fb6b3" (UID: "115fa14f-c994-4ba6-bd9e-9c47507fb6b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:48.439975 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.439770    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "115fa14f-c994-4ba6-bd9e-9c47507fb6b3" (UID: "115fa14f-c994-4ba6-bd9e-9c47507fb6b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:48.439975 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.439875    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "115fa14f-c994-4ba6-bd9e-9c47507fb6b3" (UID: "115fa14f-c994-4ba6-bd9e-9c47507fb6b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:48.440129 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.440062    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-workload-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:48.440129 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.440085    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-credential-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:48.440129 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.440102    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istiod-ca-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:48.440129 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.440117    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-workload-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:48.440291 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.440181    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-data" (OuterVolumeSpecName: "istio-data") pod "115fa14f-c994-4ba6-bd9e-9c47507fb6b3" (UID: "115fa14f-c994-4ba6-bd9e-9c47507fb6b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:48.441484 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.441463    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-token" (OuterVolumeSpecName: "istio-token") pod "115fa14f-c994-4ba6-bd9e-9c47507fb6b3" (UID: "115fa14f-c994-4ba6-bd9e-9c47507fb6b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:48.441576 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.441488    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-kube-api-access-wwxkl" (OuterVolumeSpecName: "kube-api-access-wwxkl") pod "115fa14f-c994-4ba6-bd9e-9c47507fb6b3" (UID: "115fa14f-c994-4ba6-bd9e-9c47507fb6b3"). InnerVolumeSpecName "kube-api-access-wwxkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:48.442075 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.442049    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "115fa14f-c994-4ba6-bd9e-9c47507fb6b3" (UID: "115fa14f-c994-4ba6-bd9e-9c47507fb6b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:38:48.442145 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.442128    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "115fa14f-c994-4ba6-bd9e-9c47507fb6b3" (UID: "115fa14f-c994-4ba6-bd9e-9c47507fb6b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:48.541074 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.541048    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:48.541074 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.541071    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:48.541204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.541081    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwxkl\" (UniqueName: \"kubernetes.io/projected/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-kube-api-access-wwxkl\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:48.541204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.541091    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-podinfo\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:48.541204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:48.541099    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115fa14f-c994-4ba6-bd9e-9c47507fb6b3-istio-envoy\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:38:49.279419 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:49.279391    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"
Sep 09 12:38:49.279816 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:49.279427    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz" event={"ID":"115fa14f-c994-4ba6-bd9e-9c47507fb6b3","Type":"ContainerDied","Data":"595571a61762aa623e12cc259f72b465b49a1f9abbf190f99624acf83e8a4e25"}
Sep 09 12:38:49.279816 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:49.279504    2582 scope.go:117] "RemoveContainer" containerID="f90d1dffcf3cce89b26fbd0476e2c7f9f031d86eab8a1377c897a3e6e98a264d"
Sep 09 12:38:49.299765 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:49.299740    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"]
Sep 09 12:38:49.307911 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:49.307883    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-6a8ecc-openshift-default-8c58657fb-wqlfz"]
Sep 09 12:38:50.831932 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:50.831897    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="115fa14f-c994-4ba6-bd9e-9c47507fb6b3" path="/var/lib/kubelet/pods/115fa14f-c994-4ba6-bd9e-9c47507fb6b3/volumes"
Sep 09 12:38:52.266338 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:52.266312    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:38:59.275029 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:59.274983    2582 patch_prober.go:28] interesting pod/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 12:38:59.275530 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:38:59.275053    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 12:39:00.356169 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:00.356132    2582 generic.go:356] "Generic (PLEG): container finished" podID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerID="af5d54d439493cd4c132bbb6e3e861c86a1f10bb5f2d91d999e16a2ee5015a76" exitCode=0
Sep 09 12:39:00.356590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:00.356201    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" event={"ID":"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe","Type":"ContainerDied","Data":"af5d54d439493cd4c132bbb6e3e861c86a1f10bb5f2d91d999e16a2ee5015a76"}
Sep 09 12:39:00.356590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:00.356574    2582 scope.go:117] "RemoveContainer" containerID="af5d54d439493cd4c132bbb6e3e861c86a1f10bb5f2d91d999e16a2ee5015a76"
Sep 09 12:39:00.357805 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:00.357761    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:39:00.357940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:00.357886    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:39:00.357994 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:00.357966    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:39:01.364518 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:01.364481    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" event={"ID":"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe","Type":"ContainerStarted","Data":"c891b9692e9338a0b37a01d9cd72fc4b2659a80a0ad0703b6dcead5226994257"}
Sep 09 12:39:02.308995 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:02.308917    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:39:02.309176 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:02.309150    2582 patch_prober.go:28] interesting pod/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 09 12:39:02.309230 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:02.309213    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 09 12:39:03.309864 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.309825    2582 patch_prober.go:28] interesting pod/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 09 12:39:03.310242 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.309915    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 09 12:39:03.447185 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.447151    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"]
Sep 09 12:39:03.447761 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.447741    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="115fa14f-c994-4ba6-bd9e-9c47507fb6b3" containerName="istio-proxy"
Sep 09 12:39:03.447837 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.447768    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="115fa14f-c994-4ba6-bd9e-9c47507fb6b3" containerName="istio-proxy"
Sep 09 12:39:03.447926 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.447842    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="115fa14f-c994-4ba6-bd9e-9c47507fb6b3" containerName="istio-proxy"
Sep 09 12:39:03.450687 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.450665    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.452892 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.452870    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-fe6606-openshift-default-dockercfg-9rw2z\""
Sep 09 12:39:03.462197 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.462175    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"]
Sep 09 12:39:03.501328 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.501290    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istiod-ca-cert\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.501587 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.501565    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-token\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.501738 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.501722    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-workload-socket\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.501945 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.501926    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-envoy\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.502093 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.502078    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-credential-socket\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.502236 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.502218    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-data\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.502366 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.502348    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lsfb\" (UniqueName: \"kubernetes.io/projected/e7b11f7d-e246-49fb-98eb-5ebf504618dd-kube-api-access-7lsfb\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.502524 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.502508    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-podinfo\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.502641 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.502625    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-workload-certs\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.603463 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.603383    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-envoy\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.603463 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.603424    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-credential-socket\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.603463 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.603450    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-data\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.603746 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.603471    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lsfb\" (UniqueName: \"kubernetes.io/projected/e7b11f7d-e246-49fb-98eb-5ebf504618dd-kube-api-access-7lsfb\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.603746 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.603499    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-podinfo\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.603746 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.603541    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-workload-certs\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.603746 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.603602    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istiod-ca-cert\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.603746 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.603649    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-token\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.603746 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.603702    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-workload-socket\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.604047 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.603886    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-credential-socket\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.604047 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.603916    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-data\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.604047 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.604008    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-workload-certs\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.604286 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.604267    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-workload-socket\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.604373 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.604352    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istiod-ca-cert\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.606203 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.606179    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-podinfo\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.606294 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.606204    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-envoy\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.611870 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.611828    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-token\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.611963 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.611874    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lsfb\" (UniqueName: \"kubernetes.io/projected/e7b11f7d-e246-49fb-98eb-5ebf504618dd-kube-api-access-7lsfb\") pod \"e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") " pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:03.761917 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:03.761877    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:04.075903 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:04.075834    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"]
Sep 09 12:39:04.089051 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:04.089003    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:39:04.089160 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:04.089105    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:39:04.089160 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:04.089150    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:39:04.309533 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:04.309492    2582 patch_prober.go:28] interesting pod/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 09 12:39:04.309719 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:04.309557    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 09 12:39:04.393841 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:04.393744    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9" event={"ID":"e7b11f7d-e246-49fb-98eb-5ebf504618dd","Type":"ContainerStarted","Data":"6ad1840aaa28fcb1d37a27834d198b3d30ea603ad4bc661983187315558dba51"}
Sep 09 12:39:04.393841 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:04.393819    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9" event={"ID":"e7b11f7d-e246-49fb-98eb-5ebf504618dd","Type":"ContainerStarted","Data":"7366b73eca73d21a3811cb06b0ec4a089bc86c178d937fd110db87ac793c2559"}
Sep 09 12:39:04.417083 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:04.417030    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9" podStartSLOduration=1.417007849 podStartE2EDuration="1.417007849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:39:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:39:04.415884511 +0000 UTC m=+1164.112449657" watchObservedRunningTime="2026-09-09 12:39:04.417007849 +0000 UTC m=+1164.113573073"
Sep 09 12:39:04.762633 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:04.762601    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:05.309692 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:05.309657    2582 patch_prober.go:28] interesting pod/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 09 12:39:05.309897 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:05.309717    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 09 12:39:05.767607 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:05.767579    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:06.309555 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:06.309515    2582 patch_prober.go:28] interesting pod/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 09 12:39:06.309722 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:06.309577    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 09 12:39:06.412138 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:06.412109    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:06.413129 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:06.413106    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:07.312876 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:07.312828    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:39:07.313326 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:07.313305    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:39:07.313942 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:07.313925    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:39:13.282134 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:13.282103    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"]
Sep 09 12:39:13.286616 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:13.286588    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:13.289040 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:13.289016    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-590823-serving-cert\""
Sep 09 12:39:13.299162 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:13.299135    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"]
Sep 09 12:39:13.401545 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:13.401510    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj7nr\" (UniqueName: \"kubernetes.io/projected/6737f883-79c2-47be-b5a0-17ee7ab1f905-kube-api-access-pj7nr\") pod \"maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:13.401721 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:13.401640    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6737f883-79c2-47be-b5a0-17ee7ab1f905-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:13.401721 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:13.401697    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6737f883-79c2-47be-b5a0-17ee7ab1f905-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:13.401816 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:13.401727    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6737f883-79c2-47be-b5a0-17ee7ab1f905-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:13.503018 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:13.502982    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6737f883-79c2-47be-b5a0-17ee7ab1f905-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:13.503183 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:13.503043    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6737f883-79c2-47be-b5a0-17ee7ab1f905-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:13.503183 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:13.503071    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6737f883-79c2-47be-b5a0-17ee7ab1f905-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:13.503183 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:13.503120    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj7nr\" (UniqueName: \"kubernetes.io/projected/6737f883-79c2-47be-b5a0-17ee7ab1f905-kube-api-access-pj7nr\") pod \"maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:13.503299 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:39:13.503212    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-serving-cert: secret "maas-api-e2e-shared-b-w2-590823-serving-cert" not found
Sep 09 12:39:13.503334 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:39:13.503299    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6737f883-79c2-47be-b5a0-17ee7ab1f905-maas-api-tls podName:6737f883-79c2-47be-b5a0-17ee7ab1f905 nodeName:}" failed. No retries permitted until 2026-09-09 12:39:14.003277815 +0000 UTC m=+1173.699842949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6737f883-79c2-47be-b5a0-17ee7ab1f905-maas-api-tls") pod "maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr" (UID: "6737f883-79c2-47be-b5a0-17ee7ab1f905") : secret "maas-api-e2e-shared-b-w2-590823-serving-cert" not found
Sep 09 12:39:13.503791 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:13.503767    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6737f883-79c2-47be-b5a0-17ee7ab1f905-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:13.505581 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:13.505562    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6737f883-79c2-47be-b5a0-17ee7ab1f905-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:13.513406 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:13.513384    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj7nr\" (UniqueName: \"kubernetes.io/projected/6737f883-79c2-47be-b5a0-17ee7ab1f905-kube-api-access-pj7nr\") pod \"maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:14.008873 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:14.008821    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6737f883-79c2-47be-b5a0-17ee7ab1f905-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:14.011283 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:14.011264    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6737f883-79c2-47be-b5a0-17ee7ab1f905-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:14.203614 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:14.203578    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:14.481083 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:14.481056    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"]
Sep 09 12:39:14.483217 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:39:14.483192    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6737f883_79c2_47be_b5a0_17ee7ab1f905.slice/crio-ae7843e277479725bad25f8eb41fb933888d22e80baa0e3de5e8295a1df91862 WatchSource:0}: Error finding container ae7843e277479725bad25f8eb41fb933888d22e80baa0e3de5e8295a1df91862: Status 404 returned error can't find the container with id ae7843e277479725bad25f8eb41fb933888d22e80baa0e3de5e8295a1df91862
Sep 09 12:39:15.467674 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:15.467640    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr" event={"ID":"6737f883-79c2-47be-b5a0-17ee7ab1f905","Type":"ContainerStarted","Data":"ff66d6c978b8e4d7ace03522d1ab59da452a2ba469c6d3ba2efa9b62906d3c2c"}
Sep 09 12:39:15.467875 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:15.467700    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:15.467875 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:15.467722    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr" event={"ID":"6737f883-79c2-47be-b5a0-17ee7ab1f905","Type":"ContainerStarted","Data":"ae7843e277479725bad25f8eb41fb933888d22e80baa0e3de5e8295a1df91862"}
Sep 09 12:39:15.487683 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:15.487610    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr" podStartSLOduration=2.017981725 podStartE2EDuration="2.487586106s" totalImagesPullingTime="469.604381ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:39:15.485232184 +0000 UTC m=+1175.181797353" watchObservedRunningTime="2026-09-09 12:39:15.487586106 +0000 UTC m=+1175.184151258"
Sep 09 12:39:21.482672 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:21.482641    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:39:22.453995 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:22.453956    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 12:39:22.454165 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:22.454016    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9" podUID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 12:39:24.536037 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:24.536005    2582 generic.go:356] "Generic (PLEG): container finished" podID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" containerID="6ad1840aaa28fcb1d37a27834d198b3d30ea603ad4bc661983187315558dba51" exitCode=0
Sep 09 12:39:24.536456 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:24.536085    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9" event={"ID":"e7b11f7d-e246-49fb-98eb-5ebf504618dd","Type":"ContainerDied","Data":"6ad1840aaa28fcb1d37a27834d198b3d30ea603ad4bc661983187315558dba51"}
Sep 09 12:39:24.536526 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:24.536509    2582 scope.go:117] "RemoveContainer" containerID="6ad1840aaa28fcb1d37a27834d198b3d30ea603ad4bc661983187315558dba51"
Sep 09 12:39:24.537570 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:24.537544    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:39:24.537627 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:24.537614    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:39:24.537664 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:24.537649    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:39:25.543760 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:25.543725    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9" event={"ID":"e7b11f7d-e246-49fb-98eb-5ebf504618dd","Type":"ContainerStarted","Data":"fea7a5193bbc7273f66ec6534feea6e2af0e0ae0e16009c9d38fa2df5e646b6d"}
Sep 09 12:39:25.762760 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:25.762722    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:25.763004 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:25.762978    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 09 12:39:25.763089 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:25.763047    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9" podUID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 09 12:39:26.763221 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:26.763192    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 09 12:39:26.763595 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:26.763246    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9" podUID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 09 12:39:27.763241 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:27.763205    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 09 12:39:27.763687 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:27.763263    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9" podUID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 09 12:39:28.762957 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:28.762923    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 09 12:39:28.763141 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:28.762978    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9" podUID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 09 12:39:29.762763 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:29.762729    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 09 12:39:29.763155 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:29.762788    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9" podUID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 09 12:39:30.788577 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:30.788541    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:30.789350 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:30.789324    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:30.790001 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:30.789978    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:39:37.983782 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:37.983744    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"]
Sep 09 12:39:37.988000 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:37.987977    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:37.990111 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:37.990094    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-283adb\"/\"auth-test-model-283adb-kserve-self-signed-certs\""
Sep 09 12:39:37.990190 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:37.990105    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-283adb\"/\"openshift-service-ca.crt\""
Sep 09 12:39:37.991068 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:37.991046    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-283adb\"/\"kube-root-ca.crt\""
Sep 09 12:39:37.991146 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:37.991130    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-283adb\"/\"default-dockercfg-8bz29\""
Sep 09 12:39:37.997821 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:37.997799    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"]
Sep 09 12:39:38.142909 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.142875    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05a9c986-8225-46be-aa14-02aff600e049-tls-certs\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.143076 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.142929    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-model-cache\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.143076 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.142952    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-dshm\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.143076 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.143001    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8789\" (UniqueName: \"kubernetes.io/projected/05a9c986-8225-46be-aa14-02aff600e049-kube-api-access-g8789\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.143076 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.143039    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-home\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.143224 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.143096    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-tmp-dir\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.243762 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.243734    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-tmp-dir\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.243965 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.243787    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05a9c986-8225-46be-aa14-02aff600e049-tls-certs\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.243965 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.243826    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-model-cache\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.243965 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.243844    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-dshm\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.243965 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.243903    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8789\" (UniqueName: \"kubernetes.io/projected/05a9c986-8225-46be-aa14-02aff600e049-kube-api-access-g8789\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.243965 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.243956    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-home\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.244250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.244179    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-tmp-dir\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.244250 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.244226    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-model-cache\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.244337 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.244316    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-home\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.246165 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.246144    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-dshm\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.246490 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.246472    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05a9c986-8225-46be-aa14-02aff600e049-tls-certs\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.252182 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.252164    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8789\" (UniqueName: \"kubernetes.io/projected/05a9c986-8225-46be-aa14-02aff600e049-kube-api-access-g8789\") pod \"auth-test-model-283adb-kserve-78757896d7-5zg4q\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") " pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.298798 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.298768    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:38.574361 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.574325    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"]
Sep 09 12:39:38.575175 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:39:38.575148    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05a9c986_8225_46be_aa14_02aff600e049.slice/crio-86b0c14c178d626c2ad71d6d93764b6a443c81f54e15dcac524b15280faf8972 WatchSource:0}: Error finding container 86b0c14c178d626c2ad71d6d93764b6a443c81f54e15dcac524b15280faf8972: Status 404 returned error can't find the container with id 86b0c14c178d626c2ad71d6d93764b6a443c81f54e15dcac524b15280faf8972
Sep 09 12:39:38.629212 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:38.629188    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q" event={"ID":"05a9c986-8225-46be-aa14-02aff600e049","Type":"ContainerStarted","Data":"86b0c14c178d626c2ad71d6d93764b6a443c81f54e15dcac524b15280faf8972"}
Sep 09 12:39:39.636782 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:39.636747    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q" event={"ID":"05a9c986-8225-46be-aa14-02aff600e049","Type":"ContainerStarted","Data":"3fd88c9950b8edfe669a37e6546674d1d2b7f36f2b1048857553f5f26e64817b"}
Sep 09 12:39:39.655053 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:39.655004    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q" podStartSLOduration=2.654987998 podStartE2EDuration="2.654987998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:39:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:39:39.652088151 +0000 UTC m=+1199.348653302" watchObservedRunningTime="2026-09-09 12:39:39.654987998 +0000 UTC m=+1199.351553150"
Sep 09 12:39:46.851326 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:46.851288    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"]
Sep 09 12:39:46.856684 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:46.856661    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:46.859113 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:46.859096    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6a4af97c-openshift-default-dockercfg-46v8t\""
Sep 09 12:39:46.865674 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:46.865651    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"]
Sep 09 12:39:46.926818 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:46.926791    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-istio-envoy\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:46.926977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:46.926836    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-workload-certs\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:46.926977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:46.926883    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2611395-8b49-47cb-bc32-892f354f1569-istiod-ca-cert\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:46.926977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:46.926944    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-workload-socket\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:46.926977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:46.926964    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2611395-8b49-47cb-bc32-892f354f1569-istio-token\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:46.927204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:46.927072    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-credential-socket\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:46.927204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:46.927117    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-istio-data\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:46.927204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:46.927183    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf9vx\" (UniqueName: \"kubernetes.io/projected/b2611395-8b49-47cb-bc32-892f354f1569-kube-api-access-gf9vx\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:46.927340 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:46.927211    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2611395-8b49-47cb-bc32-892f354f1569-istio-podinfo\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.027750 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.027718    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf9vx\" (UniqueName: \"kubernetes.io/projected/b2611395-8b49-47cb-bc32-892f354f1569-kube-api-access-gf9vx\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.027750 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.027754    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2611395-8b49-47cb-bc32-892f354f1569-istio-podinfo\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.027992 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.027889    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-istio-envoy\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.027992 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.027968    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-workload-certs\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.028097 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.028001    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2611395-8b49-47cb-bc32-892f354f1569-istiod-ca-cert\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.028097 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.028041    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-workload-socket\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.028097 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.028068    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2611395-8b49-47cb-bc32-892f354f1569-istio-token\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.028267 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.028138    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-credential-socket\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.028267 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.028180    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-istio-data\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.028423 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.028399    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-workload-certs\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.028527 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.028504    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-workload-socket\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.028710 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.028682    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-credential-socket\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.028780 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.028733    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-istio-data\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.029006 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.028985    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2611395-8b49-47cb-bc32-892f354f1569-istiod-ca-cert\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.030308 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.030287    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-istio-envoy\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.030443 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.030416    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2611395-8b49-47cb-bc32-892f354f1569-istio-podinfo\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.036324 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.036303    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2611395-8b49-47cb-bc32-892f354f1569-istio-token\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.036522 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.036503    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf9vx\" (UniqueName: \"kubernetes.io/projected/b2611395-8b49-47cb-bc32-892f354f1569-kube-api-access-gf9vx\") pod \"e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") " pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.169380 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.169303    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:47.449520 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.449496    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"]
Sep 09 12:39:47.450765 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:39:47.450743    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2611395_8b49_47cb_bc32_892f354f1569.slice/crio-0bfbb104a18995bf0247df776b0744db02ed4f00b5e2babe59e8356e696de38a WatchSource:0}: Error finding container 0bfbb104a18995bf0247df776b0744db02ed4f00b5e2babe59e8356e696de38a: Status 404 returned error can't find the container with id 0bfbb104a18995bf0247df776b0744db02ed4f00b5e2babe59e8356e696de38a
Sep 09 12:39:47.453496 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.453458    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:39:47.453604 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.453543    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:39:47.453604 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.453592    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:39:47.689039 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.688999    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd" event={"ID":"b2611395-8b49-47cb-bc32-892f354f1569","Type":"ContainerStarted","Data":"cf960e6cbab01035875e1bfb4332d36e0ae7dbe381dc055fd611b006c8134195"}
Sep 09 12:39:47.689220 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.689061    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd" event={"ID":"b2611395-8b49-47cb-bc32-892f354f1569","Type":"ContainerStarted","Data":"0bfbb104a18995bf0247df776b0744db02ed4f00b5e2babe59e8356e696de38a"}
Sep 09 12:39:47.711318 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:47.711226    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd" podStartSLOduration=1.711211897 podStartE2EDuration="1.711211897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:39:47.707881817 +0000 UTC m=+1207.404446966" watchObservedRunningTime="2026-09-09 12:39:47.711211897 +0000 UTC m=+1207.407777050"
Sep 09 12:39:48.170321 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:48.170288    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:48.177075 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:48.177051    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:48.299137 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:48.299109    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:48.299311 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:48.299160    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:48.311831 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:48.311801    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:39:48.696067 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:48.696035    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:48.697206 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:48.697182    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:39:48.707528 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:39:48.707504    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:40:22.932027 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:22.931993    2582 generic.go:356] "Generic (PLEG): container finished" podID="b2611395-8b49-47cb-bc32-892f354f1569" containerID="cf960e6cbab01035875e1bfb4332d36e0ae7dbe381dc055fd611b006c8134195" exitCode=0
Sep 09 12:40:22.932550 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:22.932070    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd" event={"ID":"b2611395-8b49-47cb-bc32-892f354f1569","Type":"ContainerDied","Data":"cf960e6cbab01035875e1bfb4332d36e0ae7dbe381dc055fd611b006c8134195"}
Sep 09 12:40:22.932623 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:22.932557    2582 scope.go:117] "RemoveContainer" containerID="cf960e6cbab01035875e1bfb4332d36e0ae7dbe381dc055fd611b006c8134195"
Sep 09 12:40:22.933717 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:22.933688    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:40:22.933782 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:22.933762    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:40:22.933822 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:22.933797    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:40:23.940056 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:23.940020    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd" event={"ID":"b2611395-8b49-47cb-bc32-892f354f1569","Type":"ContainerStarted","Data":"2c827c966562ef6a494708c1158035c4c780ab926e7ac6f4ce7e16aaa93b8f7e"}
Sep 09 12:40:24.178877 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:24.169683    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:40:24.178877 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:24.169716    2582 patch_prober.go:28] interesting pod/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 09 12:40:24.178877 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:24.169760    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd" podUID="b2611395-8b49-47cb-bc32-892f354f1569" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 09 12:40:25.170393 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:25.170352    2582 patch_prober.go:28] interesting pod/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 09 12:40:25.171050 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:25.171012    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd" podUID="b2611395-8b49-47cb-bc32-892f354f1569" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 09 12:40:26.170215 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:26.170178    2582 patch_prober.go:28] interesting pod/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 09 12:40:26.170509 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:26.170475    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd" podUID="b2611395-8b49-47cb-bc32-892f354f1569" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 09 12:40:27.170285 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:27.170245    2582 patch_prober.go:28] interesting pod/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 09 12:40:27.170605 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:27.170575    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd" podUID="b2611395-8b49-47cb-bc32-892f354f1569" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 09 12:40:28.170527 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:28.169982    2582 patch_prober.go:28] interesting pod/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 09 12:40:28.170527 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:28.170066    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd" podUID="b2611395-8b49-47cb-bc32-892f354f1569" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 09 12:40:29.174610 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:29.174580    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:40:29.175047 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:29.174980    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:40:29.175637 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:29.175616    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:40:50.337632 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.337552    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"]
Sep 09 12:40:50.340214 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.340198    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.342492 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.342467    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6a4af97c\"/\"default-dockercfg-6cszs\""
Sep 09 12:40:50.343109 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.343086    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6a4af97c\"/\"openshift-service-ca.crt\""
Sep 09 12:40:50.343221 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.343107    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6a4af97c\"/\"e2e-lifecycle-model-6a4af97c-kserve-self-signed-certs\""
Sep 09 12:40:50.343221 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.343130    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6a4af97c\"/\"kube-root-ca.crt\""
Sep 09 12:40:50.350453 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.350428    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"]
Sep 09 12:40:50.442624 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.442586    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-dshm\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.442783 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.442642    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cslz6\" (UniqueName: \"kubernetes.io/projected/970257f1-682a-4c3b-9571-902177db78bb-kube-api-access-cslz6\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.442783 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.442714    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/970257f1-682a-4c3b-9571-902177db78bb-tls-certs\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.442889 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.442819    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-home\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.442931 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.442919    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-model-cache\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.442968 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.442948    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-tmp-dir\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.545179 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.544334    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-model-cache\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.545179 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.544392    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-tmp-dir\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.545179 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.544457    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-dshm\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.545179 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.544504    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cslz6\" (UniqueName: \"kubernetes.io/projected/970257f1-682a-4c3b-9571-902177db78bb-kube-api-access-cslz6\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.545179 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.544538    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/970257f1-682a-4c3b-9571-902177db78bb-tls-certs\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.545179 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.544621    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-home\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.545179 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.545130    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-home\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.545695 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.545296    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-tmp-dir\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.545695 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.545489    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-model-cache\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.547703 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.547679    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-dshm\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.548435 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.548412    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/970257f1-682a-4c3b-9571-902177db78bb-tls-certs\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.553779 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.553749    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cslz6\" (UniqueName: \"kubernetes.io/projected/970257f1-682a-4c3b-9571-902177db78bb-kube-api-access-cslz6\") pod \"e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") " pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.650747 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.650664    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:40:50.926160 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:50.926135    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"]
Sep 09 12:40:50.928498 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:40:50.928474    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod970257f1_682a_4c3b_9571_902177db78bb.slice/crio-a75b0c8be7b25042cfc19eeec784193ff45696cc87378f9512b0d7bc08055962 WatchSource:0}: Error finding container a75b0c8be7b25042cfc19eeec784193ff45696cc87378f9512b0d7bc08055962: Status 404 returned error can't find the container with id a75b0c8be7b25042cfc19eeec784193ff45696cc87378f9512b0d7bc08055962
Sep 09 12:40:51.119040 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:51.119007    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg" event={"ID":"970257f1-682a-4c3b-9571-902177db78bb","Type":"ContainerStarted","Data":"a13782ed390ebf364f183f54f052b407295d3a50af951b973865c23c6cbca0fc"}
Sep 09 12:40:51.119194 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:51.119051    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg" event={"ID":"970257f1-682a-4c3b-9571-902177db78bb","Type":"ContainerStarted","Data":"a75b0c8be7b25042cfc19eeec784193ff45696cc87378f9512b0d7bc08055962"}
Sep 09 12:40:51.137662 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:40:51.137614    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg" podStartSLOduration=1.137597444 podStartE2EDuration="1.137597444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:40:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:40:51.134143354 +0000 UTC m=+1270.830708516" watchObservedRunningTime="2026-09-09 12:40:51.137597444 +0000 UTC m=+1270.834162596"
Sep 09 12:41:00.650951 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:41:00.650916    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:41:00.651402 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:41:00.650969    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:41:00.663293 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:41:00.663267    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:41:01.193747 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:41:01.193715    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:41:49.245440 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:41:49.245406    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"]
Sep 09 12:42:04.265689 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.265611    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg" podUID="970257f1-682a-4c3b-9571-902177db78bb" containerName="main" containerID="cri-o://a13782ed390ebf364f183f54f052b407295d3a50af951b973865c23c6cbca0fc" gracePeriod=45
Sep 09 12:42:04.522034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.521968    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:42:04.568737 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.568708    2582 generic.go:356] "Generic (PLEG): container finished" podID="970257f1-682a-4c3b-9571-902177db78bb" containerID="a13782ed390ebf364f183f54f052b407295d3a50af951b973865c23c6cbca0fc" exitCode=0
Sep 09 12:42:04.568984 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.568811    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"
Sep 09 12:42:04.568984 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.568836    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg" event={"ID":"970257f1-682a-4c3b-9571-902177db78bb","Type":"ContainerDied","Data":"a13782ed390ebf364f183f54f052b407295d3a50af951b973865c23c6cbca0fc"}
Sep 09 12:42:04.568984 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.568901    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg" event={"ID":"970257f1-682a-4c3b-9571-902177db78bb","Type":"ContainerDied","Data":"a75b0c8be7b25042cfc19eeec784193ff45696cc87378f9512b0d7bc08055962"}
Sep 09 12:42:04.568984 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.568923    2582 scope.go:117] "RemoveContainer" containerID="a13782ed390ebf364f183f54f052b407295d3a50af951b973865c23c6cbca0fc"
Sep 09 12:42:04.579913 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.579894    2582 scope.go:117] "RemoveContainer" containerID="a13782ed390ebf364f183f54f052b407295d3a50af951b973865c23c6cbca0fc"
Sep 09 12:42:04.580208 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:42:04.580190    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a13782ed390ebf364f183f54f052b407295d3a50af951b973865c23c6cbca0fc\": container with ID starting with a13782ed390ebf364f183f54f052b407295d3a50af951b973865c23c6cbca0fc not found: ID does not exist" containerID="a13782ed390ebf364f183f54f052b407295d3a50af951b973865c23c6cbca0fc"
Sep 09 12:42:04.580273 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.580220    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a13782ed390ebf364f183f54f052b407295d3a50af951b973865c23c6cbca0fc"} err="failed to get container status \"a13782ed390ebf364f183f54f052b407295d3a50af951b973865c23c6cbca0fc\": rpc error: code = NotFound desc = could not find container \"a13782ed390ebf364f183f54f052b407295d3a50af951b973865c23c6cbca0fc\": container with ID starting with a13782ed390ebf364f183f54f052b407295d3a50af951b973865c23c6cbca0fc not found: ID does not exist"
Sep 09 12:42:04.622872 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.622828    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/970257f1-682a-4c3b-9571-902177db78bb-tls-certs\") pod \"970257f1-682a-4c3b-9571-902177db78bb\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") "
Sep 09 12:42:04.622989 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.622901    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cslz6\" (UniqueName: \"kubernetes.io/projected/970257f1-682a-4c3b-9571-902177db78bb-kube-api-access-cslz6\") pod \"970257f1-682a-4c3b-9571-902177db78bb\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") "
Sep 09 12:42:04.622989 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.622936    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-tmp-dir\") pod \"970257f1-682a-4c3b-9571-902177db78bb\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") "
Sep 09 12:42:04.622989 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.622973    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-model-cache\") pod \"970257f1-682a-4c3b-9571-902177db78bb\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") "
Sep 09 12:42:04.623153 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.622994    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-dshm\") pod \"970257f1-682a-4c3b-9571-902177db78bb\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") "
Sep 09 12:42:04.623153 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.623069    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-home\") pod \"970257f1-682a-4c3b-9571-902177db78bb\" (UID: \"970257f1-682a-4c3b-9571-902177db78bb\") "
Sep 09 12:42:04.623265 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.623190    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "970257f1-682a-4c3b-9571-902177db78bb" (UID: "970257f1-682a-4c3b-9571-902177db78bb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:42:04.623265 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.623227    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-model-cache" (OuterVolumeSpecName: "model-cache") pod "970257f1-682a-4c3b-9571-902177db78bb" (UID: "970257f1-682a-4c3b-9571-902177db78bb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:42:04.623378 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.623358    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-home" (OuterVolumeSpecName: "home") pod "970257f1-682a-4c3b-9571-902177db78bb" (UID: "970257f1-682a-4c3b-9571-902177db78bb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:42:04.623531 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.623480    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-tmp-dir\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:42:04.623531 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.623505    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-model-cache\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:42:04.623531 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.623518    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-home\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:42:04.625152 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.625128    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-dshm" (OuterVolumeSpecName: "dshm") pod "970257f1-682a-4c3b-9571-902177db78bb" (UID: "970257f1-682a-4c3b-9571-902177db78bb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:42:04.625240 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.625176    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/970257f1-682a-4c3b-9571-902177db78bb-kube-api-access-cslz6" (OuterVolumeSpecName: "kube-api-access-cslz6") pod "970257f1-682a-4c3b-9571-902177db78bb" (UID: "970257f1-682a-4c3b-9571-902177db78bb"). InnerVolumeSpecName "kube-api-access-cslz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:42:04.625240 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.625180    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/970257f1-682a-4c3b-9571-902177db78bb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "970257f1-682a-4c3b-9571-902177db78bb" (UID: "970257f1-682a-4c3b-9571-902177db78bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:42:04.724202 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.724176    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/970257f1-682a-4c3b-9571-902177db78bb-tls-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:42:04.724202 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.724202    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cslz6\" (UniqueName: \"kubernetes.io/projected/970257f1-682a-4c3b-9571-902177db78bb-kube-api-access-cslz6\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:42:04.724352 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.724211    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/970257f1-682a-4c3b-9571-902177db78bb-dshm\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:42:04.886038 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.886014    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"]
Sep 09 12:42:04.889473 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:04.889450    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6a4af97c/e2e-lifecycle-model-6a4af97c-kserve-b94485b99-6xswg"]
Sep 09 12:42:06.825912 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:06.825880    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="970257f1-682a-4c3b-9571-902177db78bb" path="/var/lib/kubelet/pods/970257f1-682a-4c3b-9571-902177db78bb/volumes"
Sep 09 12:42:33.961892 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:33.961844    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"]
Sep 09 12:42:33.962371 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:33.962109    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd" podUID="b2611395-8b49-47cb-bc32-892f354f1569" containerName="istio-proxy" containerID="cri-o://2c827c966562ef6a494708c1158035c4c780ab926e7ac6f4ce7e16aaa93b8f7e" gracePeriod=30
Sep 09 12:42:39.216246 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.216222    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:42:39.253381 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.253361    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-istio-envoy\") pod \"b2611395-8b49-47cb-bc32-892f354f1569\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") "
Sep 09 12:42:39.253526 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.253388    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-workload-certs\") pod \"b2611395-8b49-47cb-bc32-892f354f1569\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") "
Sep 09 12:42:39.253526 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.253470    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-credential-socket\") pod \"b2611395-8b49-47cb-bc32-892f354f1569\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") "
Sep 09 12:42:39.253526 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.253513    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-workload-socket\") pod \"b2611395-8b49-47cb-bc32-892f354f1569\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") "
Sep 09 12:42:39.253698 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.253541    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2611395-8b49-47cb-bc32-892f354f1569-istio-podinfo\") pod \"b2611395-8b49-47cb-bc32-892f354f1569\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") "
Sep 09 12:42:39.253698 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.253572    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2611395-8b49-47cb-bc32-892f354f1569-istio-token\") pod \"b2611395-8b49-47cb-bc32-892f354f1569\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") "
Sep 09 12:42:39.253698 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.253642    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-istio-data\") pod \"b2611395-8b49-47cb-bc32-892f354f1569\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") "
Sep 09 12:42:39.253698 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.253669    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2611395-8b49-47cb-bc32-892f354f1569-istiod-ca-cert\") pod \"b2611395-8b49-47cb-bc32-892f354f1569\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") "
Sep 09 12:42:39.253941 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.253728    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf9vx\" (UniqueName: \"kubernetes.io/projected/b2611395-8b49-47cb-bc32-892f354f1569-kube-api-access-gf9vx\") pod \"b2611395-8b49-47cb-bc32-892f354f1569\" (UID: \"b2611395-8b49-47cb-bc32-892f354f1569\") "
Sep 09 12:42:39.253941 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.253640    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b2611395-8b49-47cb-bc32-892f354f1569" (UID: "b2611395-8b49-47cb-bc32-892f354f1569"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:42:39.253941 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.253763    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b2611395-8b49-47cb-bc32-892f354f1569" (UID: "b2611395-8b49-47cb-bc32-892f354f1569"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:42:39.253941 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.253774    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b2611395-8b49-47cb-bc32-892f354f1569" (UID: "b2611395-8b49-47cb-bc32-892f354f1569"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:42:39.254174 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.254030    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-istio-data" (OuterVolumeSpecName: "istio-data") pod "b2611395-8b49-47cb-bc32-892f354f1569" (UID: "b2611395-8b49-47cb-bc32-892f354f1569"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:42:39.254174 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.254127    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-credential-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:42:39.254174 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.254145    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-workload-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:42:39.254174 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.254159    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-istio-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:42:39.254174 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.254172    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-workload-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:42:39.254500 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.254466    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2611395-8b49-47cb-bc32-892f354f1569-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b2611395-8b49-47cb-bc32-892f354f1569" (UID: "b2611395-8b49-47cb-bc32-892f354f1569"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:42:39.256213 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.256189    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b2611395-8b49-47cb-bc32-892f354f1569" (UID: "b2611395-8b49-47cb-bc32-892f354f1569"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:42:39.256425 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.256400    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2611395-8b49-47cb-bc32-892f354f1569-istio-token" (OuterVolumeSpecName: "istio-token") pod "b2611395-8b49-47cb-bc32-892f354f1569" (UID: "b2611395-8b49-47cb-bc32-892f354f1569"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:42:39.256425 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.256407    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2611395-8b49-47cb-bc32-892f354f1569-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b2611395-8b49-47cb-bc32-892f354f1569" (UID: "b2611395-8b49-47cb-bc32-892f354f1569"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:42:39.256568 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.256520    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2611395-8b49-47cb-bc32-892f354f1569-kube-api-access-gf9vx" (OuterVolumeSpecName: "kube-api-access-gf9vx") pod "b2611395-8b49-47cb-bc32-892f354f1569" (UID: "b2611395-8b49-47cb-bc32-892f354f1569"). InnerVolumeSpecName "kube-api-access-gf9vx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:42:39.355386 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.355365    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2611395-8b49-47cb-bc32-892f354f1569-istio-podinfo\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:42:39.355386 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.355384    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2611395-8b49-47cb-bc32-892f354f1569-istio-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:42:39.355511 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.355394    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2611395-8b49-47cb-bc32-892f354f1569-istiod-ca-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:42:39.355511 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.355402    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf9vx\" (UniqueName: \"kubernetes.io/projected/b2611395-8b49-47cb-bc32-892f354f1569-kube-api-access-gf9vx\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:42:39.355511 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.355411    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2611395-8b49-47cb-bc32-892f354f1569-istio-envoy\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:42:39.781353 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.781319    2582 generic.go:356] "Generic (PLEG): container finished" podID="b2611395-8b49-47cb-bc32-892f354f1569" containerID="2c827c966562ef6a494708c1158035c4c780ab926e7ac6f4ce7e16aaa93b8f7e" exitCode=0
Sep 09 12:42:39.781543 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.781368    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd" event={"ID":"b2611395-8b49-47cb-bc32-892f354f1569","Type":"ContainerDied","Data":"2c827c966562ef6a494708c1158035c4c780ab926e7ac6f4ce7e16aaa93b8f7e"}
Sep 09 12:42:39.781543 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.781377    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"
Sep 09 12:42:39.781543 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.781431    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd" event={"ID":"b2611395-8b49-47cb-bc32-892f354f1569","Type":"ContainerDied","Data":"0bfbb104a18995bf0247df776b0744db02ed4f00b5e2babe59e8356e696de38a"}
Sep 09 12:42:39.781543 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.781462    2582 scope.go:117] "RemoveContainer" containerID="2c827c966562ef6a494708c1158035c4c780ab926e7ac6f4ce7e16aaa93b8f7e"
Sep 09 12:42:39.791519 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.791499    2582 scope.go:117] "RemoveContainer" containerID="cf960e6cbab01035875e1bfb4332d36e0ae7dbe381dc055fd611b006c8134195"
Sep 09 12:42:39.800148 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.800124    2582 scope.go:117] "RemoveContainer" containerID="2c827c966562ef6a494708c1158035c4c780ab926e7ac6f4ce7e16aaa93b8f7e"
Sep 09 12:42:39.800544 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:42:39.800520    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c827c966562ef6a494708c1158035c4c780ab926e7ac6f4ce7e16aaa93b8f7e\": container with ID starting with 2c827c966562ef6a494708c1158035c4c780ab926e7ac6f4ce7e16aaa93b8f7e not found: ID does not exist" containerID="2c827c966562ef6a494708c1158035c4c780ab926e7ac6f4ce7e16aaa93b8f7e"
Sep 09 12:42:39.800628 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.800554    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c827c966562ef6a494708c1158035c4c780ab926e7ac6f4ce7e16aaa93b8f7e"} err="failed to get container status \"2c827c966562ef6a494708c1158035c4c780ab926e7ac6f4ce7e16aaa93b8f7e\": rpc error: code = NotFound desc = could not find container \"2c827c966562ef6a494708c1158035c4c780ab926e7ac6f4ce7e16aaa93b8f7e\": container with ID starting with 2c827c966562ef6a494708c1158035c4c780ab926e7ac6f4ce7e16aaa93b8f7e not found: ID does not exist"
Sep 09 12:42:39.800628 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.800581    2582 scope.go:117] "RemoveContainer" containerID="cf960e6cbab01035875e1bfb4332d36e0ae7dbe381dc055fd611b006c8134195"
Sep 09 12:42:39.800901 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:42:39.800871    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf960e6cbab01035875e1bfb4332d36e0ae7dbe381dc055fd611b006c8134195\": container with ID starting with cf960e6cbab01035875e1bfb4332d36e0ae7dbe381dc055fd611b006c8134195 not found: ID does not exist" containerID="cf960e6cbab01035875e1bfb4332d36e0ae7dbe381dc055fd611b006c8134195"
Sep 09 12:42:39.801003 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.800907    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf960e6cbab01035875e1bfb4332d36e0ae7dbe381dc055fd611b006c8134195"} err="failed to get container status \"cf960e6cbab01035875e1bfb4332d36e0ae7dbe381dc055fd611b006c8134195\": rpc error: code = NotFound desc = could not find container \"cf960e6cbab01035875e1bfb4332d36e0ae7dbe381dc055fd611b006c8134195\": container with ID starting with cf960e6cbab01035875e1bfb4332d36e0ae7dbe381dc055fd611b006c8134195 not found: ID does not exist"
Sep 09 12:42:39.803083 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.803062    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"]
Sep 09 12:42:39.806322 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:39.806299    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6a4af97c-openshift-default-5f788ff64-zldzd"]
Sep 09 12:42:40.827402 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:40.827367    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2611395-8b49-47cb-bc32-892f354f1569" path="/var/lib/kubelet/pods/b2611395-8b49-47cb-bc32-892f354f1569/volumes"
Sep 09 12:42:52.439801 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:42:52.439767    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"]
Sep 09 12:43:07.460107 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.460070    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q" podUID="05a9c986-8225-46be-aa14-02aff600e049" containerName="main" containerID="cri-o://3fd88c9950b8edfe669a37e6546674d1d2b7f36f2b1048857553f5f26e64817b" gracePeriod=45
Sep 09 12:43:07.710331 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.710278    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:43:07.817112 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.817082    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-tmp-dir\") pod \"05a9c986-8225-46be-aa14-02aff600e049\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") "
Sep 09 12:43:07.817254 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.817122    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-dshm\") pod \"05a9c986-8225-46be-aa14-02aff600e049\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") "
Sep 09 12:43:07.817254 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.817154    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05a9c986-8225-46be-aa14-02aff600e049-tls-certs\") pod \"05a9c986-8225-46be-aa14-02aff600e049\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") "
Sep 09 12:43:07.817254 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.817169    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-home\") pod \"05a9c986-8225-46be-aa14-02aff600e049\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") "
Sep 09 12:43:07.817254 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.817222    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-model-cache\") pod \"05a9c986-8225-46be-aa14-02aff600e049\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") "
Sep 09 12:43:07.817481 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.817281    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8789\" (UniqueName: \"kubernetes.io/projected/05a9c986-8225-46be-aa14-02aff600e049-kube-api-access-g8789\") pod \"05a9c986-8225-46be-aa14-02aff600e049\" (UID: \"05a9c986-8225-46be-aa14-02aff600e049\") "
Sep 09 12:43:07.817481 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.817365    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "05a9c986-8225-46be-aa14-02aff600e049" (UID: "05a9c986-8225-46be-aa14-02aff600e049"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:43:07.817586 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.817503    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-home" (OuterVolumeSpecName: "home") pod "05a9c986-8225-46be-aa14-02aff600e049" (UID: "05a9c986-8225-46be-aa14-02aff600e049"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:43:07.817586 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.817512    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-model-cache" (OuterVolumeSpecName: "model-cache") pod "05a9c986-8225-46be-aa14-02aff600e049" (UID: "05a9c986-8225-46be-aa14-02aff600e049"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:43:07.817685 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.817668    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-home\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:43:07.817740 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.817689    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-model-cache\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:43:07.817740 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.817731    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-tmp-dir\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:43:07.819369 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.819347    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-dshm" (OuterVolumeSpecName: "dshm") pod "05a9c986-8225-46be-aa14-02aff600e049" (UID: "05a9c986-8225-46be-aa14-02aff600e049"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:43:07.819456 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.819410    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05a9c986-8225-46be-aa14-02aff600e049-kube-api-access-g8789" (OuterVolumeSpecName: "kube-api-access-g8789") pod "05a9c986-8225-46be-aa14-02aff600e049" (UID: "05a9c986-8225-46be-aa14-02aff600e049"). InnerVolumeSpecName "kube-api-access-g8789". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:43:07.819456 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.819421    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05a9c986-8225-46be-aa14-02aff600e049-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "05a9c986-8225-46be-aa14-02aff600e049" (UID: "05a9c986-8225-46be-aa14-02aff600e049"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:43:07.918366 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.918336    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8789\" (UniqueName: \"kubernetes.io/projected/05a9c986-8225-46be-aa14-02aff600e049-kube-api-access-g8789\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:43:07.918366 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.918362    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05a9c986-8225-46be-aa14-02aff600e049-dshm\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:43:07.918512 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.918376    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05a9c986-8225-46be-aa14-02aff600e049-tls-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:43:07.945426 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.945400    2582 generic.go:356] "Generic (PLEG): container finished" podID="05a9c986-8225-46be-aa14-02aff600e049" containerID="3fd88c9950b8edfe669a37e6546674d1d2b7f36f2b1048857553f5f26e64817b" exitCode=0
Sep 09 12:43:07.945547 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.945447    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q" event={"ID":"05a9c986-8225-46be-aa14-02aff600e049","Type":"ContainerDied","Data":"3fd88c9950b8edfe669a37e6546674d1d2b7f36f2b1048857553f5f26e64817b"}
Sep 09 12:43:07.945547 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.945469    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"
Sep 09 12:43:07.945547 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.945483    2582 scope.go:117] "RemoveContainer" containerID="3fd88c9950b8edfe669a37e6546674d1d2b7f36f2b1048857553f5f26e64817b"
Sep 09 12:43:07.945707 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.945473    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q" event={"ID":"05a9c986-8225-46be-aa14-02aff600e049","Type":"ContainerDied","Data":"86b0c14c178d626c2ad71d6d93764b6a443c81f54e15dcac524b15280faf8972"}
Sep 09 12:43:07.955811 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.955777    2582 scope.go:117] "RemoveContainer" containerID="3fd88c9950b8edfe669a37e6546674d1d2b7f36f2b1048857553f5f26e64817b"
Sep 09 12:43:07.956074 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:43:07.956053    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fd88c9950b8edfe669a37e6546674d1d2b7f36f2b1048857553f5f26e64817b\": container with ID starting with 3fd88c9950b8edfe669a37e6546674d1d2b7f36f2b1048857553f5f26e64817b not found: ID does not exist" containerID="3fd88c9950b8edfe669a37e6546674d1d2b7f36f2b1048857553f5f26e64817b"
Sep 09 12:43:07.956158 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.956081    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fd88c9950b8edfe669a37e6546674d1d2b7f36f2b1048857553f5f26e64817b"} err="failed to get container status \"3fd88c9950b8edfe669a37e6546674d1d2b7f36f2b1048857553f5f26e64817b\": rpc error: code = NotFound desc = could not find container \"3fd88c9950b8edfe669a37e6546674d1d2b7f36f2b1048857553f5f26e64817b\": container with ID starting with 3fd88c9950b8edfe669a37e6546674d1d2b7f36f2b1048857553f5f26e64817b not found: ID does not exist"
Sep 09 12:43:07.970626 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.970489    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"]
Sep 09 12:43:07.973283 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:07.973258    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-283adb/auth-test-model-283adb-kserve-78757896d7-5zg4q"]
Sep 09 12:43:08.827783 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:08.827753    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05a9c986-8225-46be-aa14-02aff600e049" path="/var/lib/kubelet/pods/05a9c986-8225-46be-aa14-02aff600e049/volumes"
Sep 09 12:43:29.965723 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.965687    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"]
Sep 09 12:43:29.966218 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.966202    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2611395-8b49-47cb-bc32-892f354f1569" containerName="istio-proxy"
Sep 09 12:43:29.966274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.966221    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2611395-8b49-47cb-bc32-892f354f1569" containerName="istio-proxy"
Sep 09 12:43:29.966274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.966230    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05a9c986-8225-46be-aa14-02aff600e049" containerName="main"
Sep 09 12:43:29.966274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.966237    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="05a9c986-8225-46be-aa14-02aff600e049" containerName="main"
Sep 09 12:43:29.966274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.966245    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2611395-8b49-47cb-bc32-892f354f1569" containerName="istio-proxy"
Sep 09 12:43:29.966274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.966250    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2611395-8b49-47cb-bc32-892f354f1569" containerName="istio-proxy"
Sep 09 12:43:29.966274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.966256    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="970257f1-682a-4c3b-9571-902177db78bb" containerName="main"
Sep 09 12:43:29.966274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.966260    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="970257f1-682a-4c3b-9571-902177db78bb" containerName="main"
Sep 09 12:43:29.966494 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.966323    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="970257f1-682a-4c3b-9571-902177db78bb" containerName="main"
Sep 09 12:43:29.966494 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.966330    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="05a9c986-8225-46be-aa14-02aff600e049" containerName="main"
Sep 09 12:43:29.966494 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.966339    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2611395-8b49-47cb-bc32-892f354f1569" containerName="istio-proxy"
Sep 09 12:43:29.966494 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.966345    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2611395-8b49-47cb-bc32-892f354f1569" containerName="istio-proxy"
Sep 09 12:43:29.970040 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.970012    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:29.972274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.972254    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ab8d88-openshift-default-dockercfg-cckn4\""
Sep 09 12:43:29.990229 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:29.990207    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"]
Sep 09 12:43:30.007062 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.007035    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-podinfo\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.007180 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.007077    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-workload-socket\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.007234 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.007151    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-token\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.007234 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.007224    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8252e974-bf73-4cf0-9f87-46c3d4953046-istiod-ca-cert\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.007341 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.007308    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-data\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.007399 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.007358    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qk8z\" (UniqueName: \"kubernetes.io/projected/8252e974-bf73-4cf0-9f87-46c3d4953046-kube-api-access-8qk8z\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.007453 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.007398    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-workload-certs\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.007453 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.007436    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-credential-socket\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.007599 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.007509    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-envoy\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.108199 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.108163    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-token\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.108369 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.108209    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8252e974-bf73-4cf0-9f87-46c3d4953046-istiod-ca-cert\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.108369 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.108297    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-data\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.108369 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.108342    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qk8z\" (UniqueName: \"kubernetes.io/projected/8252e974-bf73-4cf0-9f87-46c3d4953046-kube-api-access-8qk8z\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.108567 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.108379    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-workload-certs\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.108567 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.108415    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-credential-socket\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.108567 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.108514    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-envoy\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.108680 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.108576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-podinfo\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.108680 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.108598    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-workload-socket\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.108766 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.108737    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-data\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.108835 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.108781    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-workload-certs\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.108916 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.108898    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-credential-socket\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.109004 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.108983    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-workload-socket\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.109070 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.109011    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8252e974-bf73-4cf0-9f87-46c3d4953046-istiod-ca-cert\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.110933 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.110912    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-envoy\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.111294 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.111271    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-podinfo\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.116549 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.116530    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-token\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.116802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.116779    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qk8z\" (UniqueName: \"kubernetes.io/projected/8252e974-bf73-4cf0-9f87-46c3d4953046-kube-api-access-8qk8z\") pod \"e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") " pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.281285 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.281260    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:30.566063 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.566037    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"]
Sep 09 12:43:30.566664 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:43:30.566637    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8252e974_bf73_4cf0_9f87_46c3d4953046.slice/crio-91bb5eba2c4aedd49e5c4c7ba310c486988521dbba6dd2dde8633c22c8bef95c WatchSource:0}: Error finding container 91bb5eba2c4aedd49e5c4c7ba310c486988521dbba6dd2dde8633c22c8bef95c: Status 404 returned error can't find the container with id 91bb5eba2c4aedd49e5c4c7ba310c486988521dbba6dd2dde8633c22c8bef95c
Sep 09 12:43:30.568962 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.568926    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:43:30.569050 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.569001    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:43:30.569050 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:30.569039    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:43:31.084029 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:31.083996    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks" event={"ID":"8252e974-bf73-4cf0-9f87-46c3d4953046","Type":"ContainerStarted","Data":"12b19dc9d2df6f39480d2ff9d426d71df32817fd9ac0595e41a2a6c72f329339"}
Sep 09 12:43:31.084391 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:31.084045    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks" event={"ID":"8252e974-bf73-4cf0-9f87-46c3d4953046","Type":"ContainerStarted","Data":"91bb5eba2c4aedd49e5c4c7ba310c486988521dbba6dd2dde8633c22c8bef95c"}
Sep 09 12:43:31.107767 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:31.107724    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks" podStartSLOduration=2.107709295 podStartE2EDuration="2.107709295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:43:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:43:31.104477513 +0000 UTC m=+1430.801042675" watchObservedRunningTime="2026-09-09 12:43:31.107709295 +0000 UTC m=+1430.804274446"
Sep 09 12:43:31.281873 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:31.281816    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:31.287027 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:31.287006    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:32.091281 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:32.091196    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:32.092008 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:32.091988    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:36.215740 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.215701    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"]
Sep 09 12:43:36.219737 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.219719    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.222625 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.222603    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fe6606\"/\"kube-root-ca.crt\""
Sep 09 12:43:36.222745 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.222623    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fe6606\"/\"sub-test-model-fe6606-kserve-self-signed-certs\""
Sep 09 12:43:36.222745 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.222631    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fe6606\"/\"openshift-service-ca.crt\""
Sep 09 12:43:36.222882 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.222608    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fe6606\"/\"default-dockercfg-vxvmq\""
Sep 09 12:43:36.230282 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.230262    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"]
Sep 09 12:43:36.368278 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.368244    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf5tc\" (UniqueName: \"kubernetes.io/projected/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-kube-api-access-mf5tc\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.368423 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.368288    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-model-cache\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.368423 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.368347    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-dshm\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.368423 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.368383    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-tmp-dir\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.368423 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.368404    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-home\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.368423 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.368423    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-tls-certs\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.469039 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.468960    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-tmp-dir\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.469267 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.469242    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-home\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.469414 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.469281    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-tls-certs\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.469414 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.469355    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf5tc\" (UniqueName: \"kubernetes.io/projected/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-kube-api-access-mf5tc\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.469414 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.469394    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-model-cache\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.469590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.469460    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-dshm\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.469590 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.469571    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-home\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.469734 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.469391    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-tmp-dir\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.469827 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.469807    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-model-cache\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.472216 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.472152    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-dshm\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.472594 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.472568    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-tls-certs\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.479204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.479180    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf5tc\" (UniqueName: \"kubernetes.io/projected/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-kube-api-access-mf5tc\") pod \"sub-test-model-fe6606-kserve-58c7795f89-k67wh\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") " pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.531072 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.531046    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:36.807731 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:36.807703    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"]
Sep 09 12:43:36.809523 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:43:36.809500    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2eca8bb0_6b2b_46fe_b7fa_dcfa0121a43d.slice/crio-b85b5c4c33257d7bbe5050294d26273dcc62daa4dc245f153eb1406f047626ec WatchSource:0}: Error finding container b85b5c4c33257d7bbe5050294d26273dcc62daa4dc245f153eb1406f047626ec: Status 404 returned error can't find the container with id b85b5c4c33257d7bbe5050294d26273dcc62daa4dc245f153eb1406f047626ec
Sep 09 12:43:37.123288 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:37.123205    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh" event={"ID":"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d","Type":"ContainerStarted","Data":"086288cc5bd55eacf32f1503cabb9946a1eaf457989ed035108be2727dec8bc6"}
Sep 09 12:43:37.123288 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:37.123257    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh" event={"ID":"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d","Type":"ContainerStarted","Data":"b85b5c4c33257d7bbe5050294d26273dcc62daa4dc245f153eb1406f047626ec"}
Sep 09 12:43:37.144122 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:37.144071    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh" podStartSLOduration=1.144054963 podStartE2EDuration="1.144054963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:43:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:43:37.141625593 +0000 UTC m=+1436.838190743" watchObservedRunningTime="2026-09-09 12:43:37.144054963 +0000 UTC m=+1436.840620112"
Sep 09 12:43:46.531736 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:46.531703    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:46.532184 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:46.532028    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:46.544768 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:46.544747    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:47.209592 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:47.209563    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:43:53.233206 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:53.233175    2582 generic.go:356] "Generic (PLEG): container finished" podID="8252e974-bf73-4cf0-9f87-46c3d4953046" containerID="12b19dc9d2df6f39480d2ff9d426d71df32817fd9ac0595e41a2a6c72f329339" exitCode=0
Sep 09 12:43:53.233568 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:53.233246    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks" event={"ID":"8252e974-bf73-4cf0-9f87-46c3d4953046","Type":"ContainerDied","Data":"12b19dc9d2df6f39480d2ff9d426d71df32817fd9ac0595e41a2a6c72f329339"}
Sep 09 12:43:53.233637 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:53.233625    2582 scope.go:117] "RemoveContainer" containerID="12b19dc9d2df6f39480d2ff9d426d71df32817fd9ac0595e41a2a6c72f329339"
Sep 09 12:43:53.234786 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:53.234755    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:43:53.234928 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:53.234841    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:43:53.234928 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:53.234915    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:43:54.241069 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:54.241035    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks" event={"ID":"8252e974-bf73-4cf0-9f87-46c3d4953046","Type":"ContainerStarted","Data":"2218e0afcecbaa4cb527fde0f0340ca5d1dd98398bd26e8767b4c3c769918e1f"}
Sep 09 12:43:54.282331 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:54.282294    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:54.282524 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:54.282417    2582 patch_prober.go:28] interesting pod/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 09 12:43:54.282524 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:54.282473    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks" podUID="8252e974-bf73-4cf0-9f87-46c3d4953046" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 09 12:43:55.282623 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:55.282587    2582 patch_prober.go:28] interesting pod/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 09 12:43:55.283242 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:55.283207    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks" podUID="8252e974-bf73-4cf0-9f87-46c3d4953046" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 09 12:43:56.286921 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:56.286885    2582 patch_prober.go:28] interesting pod/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 09 12:43:56.287976 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:56.287941    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks" podUID="8252e974-bf73-4cf0-9f87-46c3d4953046" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 09 12:43:57.282623 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:57.282222    2582 patch_prober.go:28] interesting pod/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 09 12:43:57.282623 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:57.282281    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks" podUID="8252e974-bf73-4cf0-9f87-46c3d4953046" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 09 12:43:58.281724 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:58.281681    2582 patch_prober.go:28] interesting pod/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 09 12:43:58.282208 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:58.281745    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks" podUID="8252e974-bf73-4cf0-9f87-46c3d4953046" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 09 12:43:59.285483 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:59.285454    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:59.285872 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:59.285836    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:43:59.286465 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:43:59.286447    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:44:02.737767 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:02.737725    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"]
Sep 09 12:44:02.740580 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:02.740561    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:02.742433 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:02.742417    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-3f4a7d-serving-cert\""
Sep 09 12:44:02.752383 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:02.752357    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"]
Sep 09 12:44:02.867901 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:02.867656    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4982c8-fc35-4262-b114-97a17945e01d-maas-api-tls\") pod \"maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:02.867901 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:02.867731    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j89x9\" (UniqueName: \"kubernetes.io/projected/ea4982c8-fc35-4262-b114-97a17945e01d-kube-api-access-j89x9\") pod \"maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:02.867901 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:02.867869    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea4982c8-fc35-4262-b114-97a17945e01d-metrics-certs\") pod \"maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:02.868166 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:02.868042    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea4982c8-fc35-4262-b114-97a17945e01d-openshift-service-ca\") pod \"maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:02.969481 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:02.969451    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4982c8-fc35-4262-b114-97a17945e01d-maas-api-tls\") pod \"maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:02.969641 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:02.969488    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j89x9\" (UniqueName: \"kubernetes.io/projected/ea4982c8-fc35-4262-b114-97a17945e01d-kube-api-access-j89x9\") pod \"maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:02.969641 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:02.969516    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea4982c8-fc35-4262-b114-97a17945e01d-metrics-certs\") pod \"maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:02.969641 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:02.969537    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea4982c8-fc35-4262-b114-97a17945e01d-openshift-service-ca\") pod \"maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:02.969641 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:44:02.969608    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-serving-cert: secret "maas-api-e2e-api-b-3f4a7d-serving-cert" not found
Sep 09 12:44:02.969892 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:44:02.969687    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea4982c8-fc35-4262-b114-97a17945e01d-maas-api-tls podName:ea4982c8-fc35-4262-b114-97a17945e01d nodeName:}" failed. No retries permitted until 2026-09-09 12:44:03.469666928 +0000 UTC m=+1463.166232061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ea4982c8-fc35-4262-b114-97a17945e01d-maas-api-tls") pod "maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs" (UID: "ea4982c8-fc35-4262-b114-97a17945e01d") : secret "maas-api-e2e-api-b-3f4a7d-serving-cert" not found
Sep 09 12:44:02.970271 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:02.970214    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea4982c8-fc35-4262-b114-97a17945e01d-openshift-service-ca\") pod \"maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:02.972231 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:02.972207    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea4982c8-fc35-4262-b114-97a17945e01d-metrics-certs\") pod \"maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:02.978211 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:02.978187    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j89x9\" (UniqueName: \"kubernetes.io/projected/ea4982c8-fc35-4262-b114-97a17945e01d-kube-api-access-j89x9\") pod \"maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:03.473895 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:03.473836    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4982c8-fc35-4262-b114-97a17945e01d-maas-api-tls\") pod \"maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:03.476363 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:03.476338    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4982c8-fc35-4262-b114-97a17945e01d-maas-api-tls\") pod \"maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:03.654156 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:03.654122    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:03.940694 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:03.940568    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"]
Sep 09 12:44:03.941777 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:03.941754    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:44:04.304500 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:04.304469    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs" event={"ID":"ea4982c8-fc35-4262-b114-97a17945e01d","Type":"ContainerStarted","Data":"4365c337c2242bf19f12a9fe67948d0941883b2b2607cc963d1fd2bd427d63d0"}
Sep 09 12:44:05.311523 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:05.311491    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs" event={"ID":"ea4982c8-fc35-4262-b114-97a17945e01d","Type":"ContainerStarted","Data":"30be014797289f3e362b0865638a6a03a4b9b8898267a108edc1f04d01465e69"}
Sep 09 12:44:05.311919 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:05.311554    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:05.340546 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:05.340496    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs" podStartSLOduration=2.9320204199999997 podStartE2EDuration="3.340482268s" totalImagesPullingTime="408.461848ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:44:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:44:05.337289126 +0000 UTC m=+1465.033854288" watchObservedRunningTime="2026-09-09 12:44:05.340482268 +0000 UTC m=+1465.037047418"
Sep 09 12:44:11.322613 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:11.322582    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:44:36.539319 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:36.539286    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"]
Sep 09 12:44:45.540264 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.540231    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"]
Sep 09 12:44:45.544235 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.544214    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.546587 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.546564    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-283adb\"/\"openshift-service-ca.crt\""
Sep 09 12:44:45.555061 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.553819    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-283adb\"/\"kube-root-ca.crt\""
Sep 09 12:44:45.555061 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.554086    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-283adb\"/\"rate-test-model-283adb-kserve-self-signed-certs\""
Sep 09 12:44:45.555061 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.554281    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-283adb\"/\"default-dockercfg-8bz29\""
Sep 09 12:44:45.557153 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.556020    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"]
Sep 09 12:44:45.664648 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.664615    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-tmp-dir\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.664816 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.664689    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-dshm\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.664816 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.664717    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wngl4\" (UniqueName: \"kubernetes.io/projected/411743cc-7116-437a-a6ce-c891d954d97a-kube-api-access-wngl4\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.664816 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.664751    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/411743cc-7116-437a-a6ce-c891d954d97a-tls-certs\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.664816 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.664771    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-model-cache\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.664980 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.664875    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-home\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.765440 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.765404    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-tmp-dir\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.765619 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.765483    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-dshm\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.765619 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.765511    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wngl4\" (UniqueName: \"kubernetes.io/projected/411743cc-7116-437a-a6ce-c891d954d97a-kube-api-access-wngl4\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.765738 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.765652    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/411743cc-7116-437a-a6ce-c891d954d97a-tls-certs\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.765738 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.765686    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-model-cache\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.765738 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.765721    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-home\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.765924 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.765780    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-tmp-dir\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.766014 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.765997    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-model-cache\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.766104 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.766087    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-home\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.767770 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.767742    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-dshm\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.768156 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.768138    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/411743cc-7116-437a-a6ce-c891d954d97a-tls-certs\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.775354 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.775335    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wngl4\" (UniqueName: \"kubernetes.io/projected/411743cc-7116-437a-a6ce-c891d954d97a-kube-api-access-wngl4\") pod \"rate-test-model-283adb-kserve-5b4df85c47-pgthh\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") " pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:45.864201 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:45.864133    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:46.158597 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:46.158571    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"]
Sep 09 12:44:46.160520 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:44:46.160497    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod411743cc_7116_437a_a6ce_c891d954d97a.slice/crio-822dbf6ed11a7facfc6c1dcd3a232e1899436fb618af9cb00dba73b9d0a5ed63 WatchSource:0}: Error finding container 822dbf6ed11a7facfc6c1dcd3a232e1899436fb618af9cb00dba73b9d0a5ed63: Status 404 returned error can't find the container with id 822dbf6ed11a7facfc6c1dcd3a232e1899436fb618af9cb00dba73b9d0a5ed63
Sep 09 12:44:46.575468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:46.575428    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh" event={"ID":"411743cc-7116-437a-a6ce-c891d954d97a","Type":"ContainerStarted","Data":"473181ee16ec6b9e15c03b27d4a027175f06072fcb56f0661c023001dd35c30c"}
Sep 09 12:44:46.575949 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:46.575483    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh" event={"ID":"411743cc-7116-437a-a6ce-c891d954d97a","Type":"ContainerStarted","Data":"822dbf6ed11a7facfc6c1dcd3a232e1899436fb618af9cb00dba73b9d0a5ed63"}
Sep 09 12:44:46.595065 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:46.595020    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh" podStartSLOduration=1.5950063399999999 podStartE2EDuration="1.59500634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:44:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:44:46.592048797 +0000 UTC m=+1506.288613961" watchObservedRunningTime="2026-09-09 12:44:46.59500634 +0000 UTC m=+1506.291571491"
Sep 09 12:44:51.560133 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.560075    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh" podUID="2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d" containerName="main" containerID="cri-o://086288cc5bd55eacf32f1503cabb9946a1eaf457989ed035108be2727dec8bc6" gracePeriod=45
Sep 09 12:44:51.821874 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.821792    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:44:51.924924 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.924896    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf5tc\" (UniqueName: \"kubernetes.io/projected/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-kube-api-access-mf5tc\") pod \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") "
Sep 09 12:44:51.925106 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.924933    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-model-cache\") pod \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") "
Sep 09 12:44:51.925106 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.924952    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-home\") pod \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") "
Sep 09 12:44:51.925106 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.924972    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-tmp-dir\") pod \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") "
Sep 09 12:44:51.925106 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.924991    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-tls-certs\") pod \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") "
Sep 09 12:44:51.925106 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.925041    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-dshm\") pod \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\" (UID: \"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d\") "
Sep 09 12:44:51.925367 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.925173    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d" (UID: "2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:44:51.925367 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.925201    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-model-cache" (OuterVolumeSpecName: "model-cache") pod "2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d" (UID: "2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:44:51.925367 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.925214    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-home" (OuterVolumeSpecName: "home") pod "2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d" (UID: "2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:44:51.925499 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.925470    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-model-cache\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:44:51.925499 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.925488    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-home\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:44:51.925499 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.925496    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-tmp-dir\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:44:51.927329 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.927306    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-dshm" (OuterVolumeSpecName: "dshm") pod "2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d" (UID: "2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:44:51.927416 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.927366    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-kube-api-access-mf5tc" (OuterVolumeSpecName: "kube-api-access-mf5tc") pod "2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d" (UID: "2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d"). InnerVolumeSpecName "kube-api-access-mf5tc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:44:51.927416 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:51.927386    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d" (UID: "2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:44:52.026988 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:52.026961    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf5tc\" (UniqueName: \"kubernetes.io/projected/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-kube-api-access-mf5tc\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:44:52.026988 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:52.026983    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-tls-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:44:52.026988 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:52.026992    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d-dshm\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:44:52.615404 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:52.615369    2582 generic.go:356] "Generic (PLEG): container finished" podID="2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d" containerID="086288cc5bd55eacf32f1503cabb9946a1eaf457989ed035108be2727dec8bc6" exitCode=0
Sep 09 12:44:52.615799 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:52.615412    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"
Sep 09 12:44:52.615799 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:52.615443    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh" event={"ID":"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d","Type":"ContainerDied","Data":"086288cc5bd55eacf32f1503cabb9946a1eaf457989ed035108be2727dec8bc6"}
Sep 09 12:44:52.615799 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:52.615483    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh" event={"ID":"2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d","Type":"ContainerDied","Data":"b85b5c4c33257d7bbe5050294d26273dcc62daa4dc245f153eb1406f047626ec"}
Sep 09 12:44:52.615799 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:52.615500    2582 scope.go:117] "RemoveContainer" containerID="086288cc5bd55eacf32f1503cabb9946a1eaf457989ed035108be2727dec8bc6"
Sep 09 12:44:52.625529 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:52.625515    2582 scope.go:117] "RemoveContainer" containerID="086288cc5bd55eacf32f1503cabb9946a1eaf457989ed035108be2727dec8bc6"
Sep 09 12:44:52.625784 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:44:52.625766    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"086288cc5bd55eacf32f1503cabb9946a1eaf457989ed035108be2727dec8bc6\": container with ID starting with 086288cc5bd55eacf32f1503cabb9946a1eaf457989ed035108be2727dec8bc6 not found: ID does not exist" containerID="086288cc5bd55eacf32f1503cabb9946a1eaf457989ed035108be2727dec8bc6"
Sep 09 12:44:52.625884 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:52.625798    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"086288cc5bd55eacf32f1503cabb9946a1eaf457989ed035108be2727dec8bc6"} err="failed to get container status \"086288cc5bd55eacf32f1503cabb9946a1eaf457989ed035108be2727dec8bc6\": rpc error: code = NotFound desc = could not find container \"086288cc5bd55eacf32f1503cabb9946a1eaf457989ed035108be2727dec8bc6\": container with ID starting with 086288cc5bd55eacf32f1503cabb9946a1eaf457989ed035108be2727dec8bc6 not found: ID does not exist"
Sep 09 12:44:52.637214 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:52.637189    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"]
Sep 09 12:44:52.640763 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:52.640745    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fe6606/sub-test-model-fe6606-kserve-58c7795f89-k67wh"]
Sep 09 12:44:52.766708 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:52.766678    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"]
Sep 09 12:44:52.766970 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:52.766944    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks" podUID="8252e974-bf73-4cf0-9f87-46c3d4953046" containerName="istio-proxy" containerID="cri-o://2218e0afcecbaa4cb527fde0f0340ca5d1dd98398bd26e8767b4c3c769918e1f" gracePeriod=30
Sep 09 12:44:52.825101 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:52.825067    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d" path="/var/lib/kubelet/pods/2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d/volumes"
Sep 09 12:44:55.865248 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:55.865215    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:55.865692 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:55.865498    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:55.877568 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:55.877544    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:56.661562 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:56.661538    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:44:58.023953 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.023922    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:44:58.183651 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.183583    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-workload-certs\") pod \"8252e974-bf73-4cf0-9f87-46c3d4953046\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") "
Sep 09 12:44:58.183651 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.183626    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qk8z\" (UniqueName: \"kubernetes.io/projected/8252e974-bf73-4cf0-9f87-46c3d4953046-kube-api-access-8qk8z\") pod \"8252e974-bf73-4cf0-9f87-46c3d4953046\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") "
Sep 09 12:44:58.183869 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.183653    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8252e974-bf73-4cf0-9f87-46c3d4953046-istiod-ca-cert\") pod \"8252e974-bf73-4cf0-9f87-46c3d4953046\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") "
Sep 09 12:44:58.183869 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.183679    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-workload-socket\") pod \"8252e974-bf73-4cf0-9f87-46c3d4953046\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") "
Sep 09 12:44:58.183869 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.183704    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-credential-socket\") pod \"8252e974-bf73-4cf0-9f87-46c3d4953046\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") "
Sep 09 12:44:58.183869 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.183814    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-envoy\") pod \"8252e974-bf73-4cf0-9f87-46c3d4953046\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") "
Sep 09 12:44:58.184082 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.183878    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-podinfo\") pod \"8252e974-bf73-4cf0-9f87-46c3d4953046\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") "
Sep 09 12:44:58.184082 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.183896    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8252e974-bf73-4cf0-9f87-46c3d4953046" (UID: "8252e974-bf73-4cf0-9f87-46c3d4953046"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:44:58.184082 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.183909    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-token\") pod \"8252e974-bf73-4cf0-9f87-46c3d4953046\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") "
Sep 09 12:44:58.184082 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.183947    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-data\") pod \"8252e974-bf73-4cf0-9f87-46c3d4953046\" (UID: \"8252e974-bf73-4cf0-9f87-46c3d4953046\") "
Sep 09 12:44:58.184082 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.184008    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8252e974-bf73-4cf0-9f87-46c3d4953046" (UID: "8252e974-bf73-4cf0-9f87-46c3d4953046"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:44:58.184326 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.184091    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8252e974-bf73-4cf0-9f87-46c3d4953046" (UID: "8252e974-bf73-4cf0-9f87-46c3d4953046"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:44:58.184326 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.184123    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8252e974-bf73-4cf0-9f87-46c3d4953046-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8252e974-bf73-4cf0-9f87-46c3d4953046" (UID: "8252e974-bf73-4cf0-9f87-46c3d4953046"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:44:58.184326 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.184279    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-data" (OuterVolumeSpecName: "istio-data") pod "8252e974-bf73-4cf0-9f87-46c3d4953046" (UID: "8252e974-bf73-4cf0-9f87-46c3d4953046"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:44:58.184469 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.184454    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-workload-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:44:58.184525 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.184474    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8252e974-bf73-4cf0-9f87-46c3d4953046-istiod-ca-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:44:58.184525 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.184489    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-workload-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:44:58.184525 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.184503    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-credential-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:44:58.184525 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.184517    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:44:58.185883 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.185841    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8252e974-bf73-4cf0-9f87-46c3d4953046-kube-api-access-8qk8z" (OuterVolumeSpecName: "kube-api-access-8qk8z") pod "8252e974-bf73-4cf0-9f87-46c3d4953046" (UID: "8252e974-bf73-4cf0-9f87-46c3d4953046"). InnerVolumeSpecName "kube-api-access-8qk8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:44:58.186043 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.186000    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8252e974-bf73-4cf0-9f87-46c3d4953046" (UID: "8252e974-bf73-4cf0-9f87-46c3d4953046"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:44:58.186279 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.186253    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-token" (OuterVolumeSpecName: "istio-token") pod "8252e974-bf73-4cf0-9f87-46c3d4953046" (UID: "8252e974-bf73-4cf0-9f87-46c3d4953046"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:44:58.186406 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.186392    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8252e974-bf73-4cf0-9f87-46c3d4953046" (UID: "8252e974-bf73-4cf0-9f87-46c3d4953046"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:44:58.285386 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.285366    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qk8z\" (UniqueName: \"kubernetes.io/projected/8252e974-bf73-4cf0-9f87-46c3d4953046-kube-api-access-8qk8z\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:44:58.285386 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.285386    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-envoy\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:44:58.285519 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.285394    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-podinfo\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:44:58.285519 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.285402    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8252e974-bf73-4cf0-9f87-46c3d4953046-istio-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:44:58.664990 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.664954    2582 generic.go:356] "Generic (PLEG): container finished" podID="8252e974-bf73-4cf0-9f87-46c3d4953046" containerID="2218e0afcecbaa4cb527fde0f0340ca5d1dd98398bd26e8767b4c3c769918e1f" exitCode=0
Sep 09 12:44:58.665164 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.665000    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"
Sep 09 12:44:58.665164 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.665042    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks" event={"ID":"8252e974-bf73-4cf0-9f87-46c3d4953046","Type":"ContainerDied","Data":"2218e0afcecbaa4cb527fde0f0340ca5d1dd98398bd26e8767b4c3c769918e1f"}
Sep 09 12:44:58.665164 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.665096    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks" event={"ID":"8252e974-bf73-4cf0-9f87-46c3d4953046","Type":"ContainerDied","Data":"91bb5eba2c4aedd49e5c4c7ba310c486988521dbba6dd2dde8633c22c8bef95c"}
Sep 09 12:44:58.665164 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.665121    2582 scope.go:117] "RemoveContainer" containerID="2218e0afcecbaa4cb527fde0f0340ca5d1dd98398bd26e8767b4c3c769918e1f"
Sep 09 12:44:58.677248 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.677232    2582 scope.go:117] "RemoveContainer" containerID="12b19dc9d2df6f39480d2ff9d426d71df32817fd9ac0595e41a2a6c72f329339"
Sep 09 12:44:58.686224 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.686201    2582 scope.go:117] "RemoveContainer" containerID="2218e0afcecbaa4cb527fde0f0340ca5d1dd98398bd26e8767b4c3c769918e1f"
Sep 09 12:44:58.686557 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:44:58.686536    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2218e0afcecbaa4cb527fde0f0340ca5d1dd98398bd26e8767b4c3c769918e1f\": container with ID starting with 2218e0afcecbaa4cb527fde0f0340ca5d1dd98398bd26e8767b4c3c769918e1f not found: ID does not exist" containerID="2218e0afcecbaa4cb527fde0f0340ca5d1dd98398bd26e8767b4c3c769918e1f"
Sep 09 12:44:58.686642 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.686565    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2218e0afcecbaa4cb527fde0f0340ca5d1dd98398bd26e8767b4c3c769918e1f"} err="failed to get container status \"2218e0afcecbaa4cb527fde0f0340ca5d1dd98398bd26e8767b4c3c769918e1f\": rpc error: code = NotFound desc = could not find container \"2218e0afcecbaa4cb527fde0f0340ca5d1dd98398bd26e8767b4c3c769918e1f\": container with ID starting with 2218e0afcecbaa4cb527fde0f0340ca5d1dd98398bd26e8767b4c3c769918e1f not found: ID does not exist"
Sep 09 12:44:58.686642 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.686587    2582 scope.go:117] "RemoveContainer" containerID="12b19dc9d2df6f39480d2ff9d426d71df32817fd9ac0595e41a2a6c72f329339"
Sep 09 12:44:58.686994 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:44:58.686961    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12b19dc9d2df6f39480d2ff9d426d71df32817fd9ac0595e41a2a6c72f329339\": container with ID starting with 12b19dc9d2df6f39480d2ff9d426d71df32817fd9ac0595e41a2a6c72f329339 not found: ID does not exist" containerID="12b19dc9d2df6f39480d2ff9d426d71df32817fd9ac0595e41a2a6c72f329339"
Sep 09 12:44:58.687099 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.686995    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12b19dc9d2df6f39480d2ff9d426d71df32817fd9ac0595e41a2a6c72f329339"} err="failed to get container status \"12b19dc9d2df6f39480d2ff9d426d71df32817fd9ac0595e41a2a6c72f329339\": rpc error: code = NotFound desc = could not find container \"12b19dc9d2df6f39480d2ff9d426d71df32817fd9ac0595e41a2a6c72f329339\": container with ID starting with 12b19dc9d2df6f39480d2ff9d426d71df32817fd9ac0595e41a2a6c72f329339 not found: ID does not exist"
Sep 09 12:44:58.688435 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.688417    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"]
Sep 09 12:44:58.692788 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.692769    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ab8d88-openshift-default-7ff5bb4466-q2fks"]
Sep 09 12:44:58.826115 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:44:58.826088    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8252e974-bf73-4cf0-9f87-46c3d4953046" path="/var/lib/kubelet/pods/8252e974-bf73-4cf0-9f87-46c3d4953046/volumes"
Sep 09 12:45:08.700141 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:08.700053    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"]
Sep 09 12:45:08.700584 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:08.700303    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs" podUID="ea4982c8-fc35-4262-b114-97a17945e01d" containerName="maas-api" containerID="cri-o://30be014797289f3e362b0865638a6a03a4b9b8898267a108edc1f04d01465e69" gracePeriod=30
Sep 09 12:45:08.955473 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:08.955414    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:45:09.091430 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.091402    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea4982c8-fc35-4262-b114-97a17945e01d-metrics-certs\") pod \"ea4982c8-fc35-4262-b114-97a17945e01d\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") "
Sep 09 12:45:09.091430 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.091437    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea4982c8-fc35-4262-b114-97a17945e01d-openshift-service-ca\") pod \"ea4982c8-fc35-4262-b114-97a17945e01d\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") "
Sep 09 12:45:09.091689 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.091457    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j89x9\" (UniqueName: \"kubernetes.io/projected/ea4982c8-fc35-4262-b114-97a17945e01d-kube-api-access-j89x9\") pod \"ea4982c8-fc35-4262-b114-97a17945e01d\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") "
Sep 09 12:45:09.091689 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.091544    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4982c8-fc35-4262-b114-97a17945e01d-maas-api-tls\") pod \"ea4982c8-fc35-4262-b114-97a17945e01d\" (UID: \"ea4982c8-fc35-4262-b114-97a17945e01d\") "
Sep 09 12:45:09.092045 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.092009    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea4982c8-fc35-4262-b114-97a17945e01d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ea4982c8-fc35-4262-b114-97a17945e01d" (UID: "ea4982c8-fc35-4262-b114-97a17945e01d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:45:09.094033 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.094005    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea4982c8-fc35-4262-b114-97a17945e01d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ea4982c8-fc35-4262-b114-97a17945e01d" (UID: "ea4982c8-fc35-4262-b114-97a17945e01d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:45:09.094223 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.094203    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea4982c8-fc35-4262-b114-97a17945e01d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ea4982c8-fc35-4262-b114-97a17945e01d" (UID: "ea4982c8-fc35-4262-b114-97a17945e01d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:45:09.094332 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.094307    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea4982c8-fc35-4262-b114-97a17945e01d-kube-api-access-j89x9" (OuterVolumeSpecName: "kube-api-access-j89x9") pod "ea4982c8-fc35-4262-b114-97a17945e01d" (UID: "ea4982c8-fc35-4262-b114-97a17945e01d"). InnerVolumeSpecName "kube-api-access-j89x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:09.192609 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.192576    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea4982c8-fc35-4262-b114-97a17945e01d-metrics-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:09.192609 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.192607    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea4982c8-fc35-4262-b114-97a17945e01d-openshift-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:09.192771 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.192624    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j89x9\" (UniqueName: \"kubernetes.io/projected/ea4982c8-fc35-4262-b114-97a17945e01d-kube-api-access-j89x9\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:09.192771 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.192637    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea4982c8-fc35-4262-b114-97a17945e01d-maas-api-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:09.737310 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.737280    2582 generic.go:356] "Generic (PLEG): container finished" podID="ea4982c8-fc35-4262-b114-97a17945e01d" containerID="30be014797289f3e362b0865638a6a03a4b9b8898267a108edc1f04d01465e69" exitCode=0
Sep 09 12:45:09.737729 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.737397    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"
Sep 09 12:45:09.737729 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.737435    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs" event={"ID":"ea4982c8-fc35-4262-b114-97a17945e01d","Type":"ContainerDied","Data":"30be014797289f3e362b0865638a6a03a4b9b8898267a108edc1f04d01465e69"}
Sep 09 12:45:09.737729 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.737487    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs" event={"ID":"ea4982c8-fc35-4262-b114-97a17945e01d","Type":"ContainerDied","Data":"4365c337c2242bf19f12a9fe67948d0941883b2b2607cc963d1fd2bd427d63d0"}
Sep 09 12:45:09.737729 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.737511    2582 scope.go:117] "RemoveContainer" containerID="30be014797289f3e362b0865638a6a03a4b9b8898267a108edc1f04d01465e69"
Sep 09 12:45:09.747830 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.747805    2582 scope.go:117] "RemoveContainer" containerID="30be014797289f3e362b0865638a6a03a4b9b8898267a108edc1f04d01465e69"
Sep 09 12:45:09.748147 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:45:09.748124    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30be014797289f3e362b0865638a6a03a4b9b8898267a108edc1f04d01465e69\": container with ID starting with 30be014797289f3e362b0865638a6a03a4b9b8898267a108edc1f04d01465e69 not found: ID does not exist" containerID="30be014797289f3e362b0865638a6a03a4b9b8898267a108edc1f04d01465e69"
Sep 09 12:45:09.748224 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.748155    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30be014797289f3e362b0865638a6a03a4b9b8898267a108edc1f04d01465e69"} err="failed to get container status \"30be014797289f3e362b0865638a6a03a4b9b8898267a108edc1f04d01465e69\": rpc error: code = NotFound desc = could not find container \"30be014797289f3e362b0865638a6a03a4b9b8898267a108edc1f04d01465e69\": container with ID starting with 30be014797289f3e362b0865638a6a03a4b9b8898267a108edc1f04d01465e69 not found: ID does not exist"
Sep 09 12:45:09.759417 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.759387    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"]
Sep 09 12:45:09.763093 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:09.763067    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3f4a7d-5f58dc7f58-nckjs"]
Sep 09 12:45:10.825531 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:10.825497    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea4982c8-fc35-4262-b114-97a17945e01d" path="/var/lib/kubelet/pods/ea4982c8-fc35-4262-b114-97a17945e01d/volumes"
Sep 09 12:45:26.846626 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:26.846595    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"]
Sep 09 12:45:26.847214 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:26.846794    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz" podUID="b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38" containerName="maas-api" containerID="cri-o://61165a6f11aefdd64a740cf673b7e133159feca85a111e01b341bec8b8987af8" gracePeriod=30
Sep 09 12:45:27.107254 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.107193    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:45:27.157122 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.157082    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-openshift-service-ca\") pod \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") "
Sep 09 12:45:27.157326 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.157262    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-metrics-certs\") pod \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") "
Sep 09 12:45:27.157326 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.157300    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5nxs\" (UniqueName: \"kubernetes.io/projected/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-kube-api-access-r5nxs\") pod \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") "
Sep 09 12:45:27.157467 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.157326    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-maas-api-tls\") pod \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\" (UID: \"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38\") "
Sep 09 12:45:27.157525 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.157498    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38" (UID: "b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:45:27.157692 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.157673    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-openshift-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:27.159527 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.159479    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38" (UID: "b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:45:27.159632 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.159529    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38" (UID: "b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:45:27.159632 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.159554    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-kube-api-access-r5nxs" (OuterVolumeSpecName: "kube-api-access-r5nxs") pod "b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38" (UID: "b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38"). InnerVolumeSpecName "kube-api-access-r5nxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:27.258803 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.258748    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-metrics-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:27.258803 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.258791    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5nxs\" (UniqueName: \"kubernetes.io/projected/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-kube-api-access-r5nxs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:27.258803 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.258808    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38-maas-api-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:27.849516 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.849480    2582 generic.go:356] "Generic (PLEG): container finished" podID="b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38" containerID="61165a6f11aefdd64a740cf673b7e133159feca85a111e01b341bec8b8987af8" exitCode=0
Sep 09 12:45:27.849977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.849560    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"
Sep 09 12:45:27.849977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.849579    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz" event={"ID":"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38","Type":"ContainerDied","Data":"61165a6f11aefdd64a740cf673b7e133159feca85a111e01b341bec8b8987af8"}
Sep 09 12:45:27.849977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.849653    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz" event={"ID":"b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38","Type":"ContainerDied","Data":"e57adddba6707e4726752841b557bb5da43d96847fbf76e3e4e67cf0ffcb8493"}
Sep 09 12:45:27.849977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.849685    2582 scope.go:117] "RemoveContainer" containerID="61165a6f11aefdd64a740cf673b7e133159feca85a111e01b341bec8b8987af8"
Sep 09 12:45:27.859697 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.859680    2582 scope.go:117] "RemoveContainer" containerID="61165a6f11aefdd64a740cf673b7e133159feca85a111e01b341bec8b8987af8"
Sep 09 12:45:27.860083 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:45:27.860045    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61165a6f11aefdd64a740cf673b7e133159feca85a111e01b341bec8b8987af8\": container with ID starting with 61165a6f11aefdd64a740cf673b7e133159feca85a111e01b341bec8b8987af8 not found: ID does not exist" containerID="61165a6f11aefdd64a740cf673b7e133159feca85a111e01b341bec8b8987af8"
Sep 09 12:45:27.860161 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.860078    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61165a6f11aefdd64a740cf673b7e133159feca85a111e01b341bec8b8987af8"} err="failed to get container status \"61165a6f11aefdd64a740cf673b7e133159feca85a111e01b341bec8b8987af8\": rpc error: code = NotFound desc = could not find container \"61165a6f11aefdd64a740cf673b7e133159feca85a111e01b341bec8b8987af8\": container with ID starting with 61165a6f11aefdd64a740cf673b7e133159feca85a111e01b341bec8b8987af8 not found: ID does not exist"
Sep 09 12:45:27.874724 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.874689    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"]
Sep 09 12:45:27.879369 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:27.879343    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a3f43f-dd7d7bfb4-x5xtz"]
Sep 09 12:45:28.826978 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:28.826941    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38" path="/var/lib/kubelet/pods/b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38/volumes"
Sep 09 12:45:34.373371 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:34.373340    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"]
Sep 09 12:45:34.373834 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:34.373602    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerName="istio-proxy" containerID="cri-o://c891b9692e9338a0b37a01d9cd72fc4b2659a80a0ad0703b6dcead5226994257" gracePeriod=30
Sep 09 12:45:37.315070 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:37.315028    2582 patch_prober.go:28] interesting pod/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 12:45:37.315427 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:37.315086    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 12:45:39.632713 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.632688    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:45:39.783672 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.783638    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-credential-socket\") pod \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") "
Sep 09 12:45:39.783672 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.783676    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-workload-socket\") pod \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") "
Sep 09 12:45:39.783953 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.783727    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-data\") pod \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") "
Sep 09 12:45:39.783953 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.783747    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istiod-ca-cert\") pod \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") "
Sep 09 12:45:39.783953 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.783794    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-token\") pod \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") "
Sep 09 12:45:39.783953 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.783842    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-workload-certs\") pod \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") "
Sep 09 12:45:39.783953 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.783899    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-podinfo\") pod \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") "
Sep 09 12:45:39.783953 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.783931    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rt2q7\" (UniqueName: \"kubernetes.io/projected/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-kube-api-access-rt2q7\") pod \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") "
Sep 09 12:45:39.784246 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.783977    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" (UID: "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:39.784246 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.783988    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-envoy\") pod \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\" (UID: \"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe\") "
Sep 09 12:45:39.784246 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.784099    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-data" (OuterVolumeSpecName: "istio-data") pod "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" (UID: "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:39.784246 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.784109    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" (UID: "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:39.784246 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.784175    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" (UID: "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:45:39.784246 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.784207    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" (UID: "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:39.784500 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.784435    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-credential-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:39.784500 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.784457    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-workload-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:39.784500 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.784473    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:39.784500 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.784486    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istiod-ca-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:39.784500 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.784499    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-workload-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:39.786169 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.786141    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-kube-api-access-rt2q7" (OuterVolumeSpecName: "kube-api-access-rt2q7") pod "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" (UID: "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe"). InnerVolumeSpecName "kube-api-access-rt2q7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:39.786291 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.786215    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" (UID: "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:45:39.786560 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.786543    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" (UID: "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:39.786560 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.786543    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-token" (OuterVolumeSpecName: "istio-token") pod "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" (UID: "9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:39.885244 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.885218    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-podinfo\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:39.885244 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.885242    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt2q7\" (UniqueName: \"kubernetes.io/projected/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-kube-api-access-rt2q7\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:39.885421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.885254    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-envoy\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:39.885421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.885262    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe-istio-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:39.923406 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.923373    2582 generic.go:356] "Generic (PLEG): container finished" podID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerID="c891b9692e9338a0b37a01d9cd72fc4b2659a80a0ad0703b6dcead5226994257" exitCode=0
Sep 09 12:45:39.923533 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.923430    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"
Sep 09 12:45:39.923533 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.923462    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" event={"ID":"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe","Type":"ContainerDied","Data":"c891b9692e9338a0b37a01d9cd72fc4b2659a80a0ad0703b6dcead5226994257"}
Sep 09 12:45:39.923533 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.923522    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5" event={"ID":"9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe","Type":"ContainerDied","Data":"f53f18e06cd0b9b3d078f1a05b620c2cc80cdff1ab9aa0ee92e8f8219b12f110"}
Sep 09 12:45:39.923638 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.923546    2582 scope.go:117] "RemoveContainer" containerID="c891b9692e9338a0b37a01d9cd72fc4b2659a80a0ad0703b6dcead5226994257"
Sep 09 12:45:39.933554 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.933537    2582 scope.go:117] "RemoveContainer" containerID="af5d54d439493cd4c132bbb6e3e861c86a1f10bb5f2d91d999e16a2ee5015a76"
Sep 09 12:45:39.941728 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.941713    2582 scope.go:117] "RemoveContainer" containerID="c891b9692e9338a0b37a01d9cd72fc4b2659a80a0ad0703b6dcead5226994257"
Sep 09 12:45:39.942100 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:45:39.942069    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c891b9692e9338a0b37a01d9cd72fc4b2659a80a0ad0703b6dcead5226994257\": container with ID starting with c891b9692e9338a0b37a01d9cd72fc4b2659a80a0ad0703b6dcead5226994257 not found: ID does not exist" containerID="c891b9692e9338a0b37a01d9cd72fc4b2659a80a0ad0703b6dcead5226994257"
Sep 09 12:45:39.942260 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.942227    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c891b9692e9338a0b37a01d9cd72fc4b2659a80a0ad0703b6dcead5226994257"} err="failed to get container status \"c891b9692e9338a0b37a01d9cd72fc4b2659a80a0ad0703b6dcead5226994257\": rpc error: code = NotFound desc = could not find container \"c891b9692e9338a0b37a01d9cd72fc4b2659a80a0ad0703b6dcead5226994257\": container with ID starting with c891b9692e9338a0b37a01d9cd72fc4b2659a80a0ad0703b6dcead5226994257 not found: ID does not exist"
Sep 09 12:45:39.942260 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.942260    2582 scope.go:117] "RemoveContainer" containerID="af5d54d439493cd4c132bbb6e3e861c86a1f10bb5f2d91d999e16a2ee5015a76"
Sep 09 12:45:39.942719 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:45:39.942697    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af5d54d439493cd4c132bbb6e3e861c86a1f10bb5f2d91d999e16a2ee5015a76\": container with ID starting with af5d54d439493cd4c132bbb6e3e861c86a1f10bb5f2d91d999e16a2ee5015a76 not found: ID does not exist" containerID="af5d54d439493cd4c132bbb6e3e861c86a1f10bb5f2d91d999e16a2ee5015a76"
Sep 09 12:45:39.942802 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.942728    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af5d54d439493cd4c132bbb6e3e861c86a1f10bb5f2d91d999e16a2ee5015a76"} err="failed to get container status \"af5d54d439493cd4c132bbb6e3e861c86a1f10bb5f2d91d999e16a2ee5015a76\": rpc error: code = NotFound desc = could not find container \"af5d54d439493cd4c132bbb6e3e861c86a1f10bb5f2d91d999e16a2ee5015a76\": container with ID starting with af5d54d439493cd4c132bbb6e3e861c86a1f10bb5f2d91d999e16a2ee5015a76 not found: ID does not exist"
Sep 09 12:45:39.945496 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.945477    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"]
Sep 09 12:45:39.950750 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:39.950726    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a3f43f-openshift-default-6b9599474d-x6kg5"]
Sep 09 12:45:40.825541 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:40.825498    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" path="/var/lib/kubelet/pods/9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe/volumes"
Sep 09 12:45:45.293994 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.293959    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"]
Sep 09 12:45:45.294445 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.294187    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr" podUID="6737f883-79c2-47be-b5a0-17ee7ab1f905" containerName="maas-api" containerID="cri-o://ff66d6c978b8e4d7ace03522d1ab59da452a2ba469c6d3ba2efa9b62906d3c2c" gracePeriod=30
Sep 09 12:45:45.553803 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.553745    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:45:45.737364 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.737335    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pj7nr\" (UniqueName: \"kubernetes.io/projected/6737f883-79c2-47be-b5a0-17ee7ab1f905-kube-api-access-pj7nr\") pod \"6737f883-79c2-47be-b5a0-17ee7ab1f905\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") "
Sep 09 12:45:45.737527 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.737376    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6737f883-79c2-47be-b5a0-17ee7ab1f905-openshift-service-ca\") pod \"6737f883-79c2-47be-b5a0-17ee7ab1f905\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") "
Sep 09 12:45:45.737527 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.737424    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6737f883-79c2-47be-b5a0-17ee7ab1f905-maas-api-tls\") pod \"6737f883-79c2-47be-b5a0-17ee7ab1f905\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") "
Sep 09 12:45:45.737527 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.737492    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6737f883-79c2-47be-b5a0-17ee7ab1f905-metrics-certs\") pod \"6737f883-79c2-47be-b5a0-17ee7ab1f905\" (UID: \"6737f883-79c2-47be-b5a0-17ee7ab1f905\") "
Sep 09 12:45:45.737757 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.737734    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6737f883-79c2-47be-b5a0-17ee7ab1f905-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6737f883-79c2-47be-b5a0-17ee7ab1f905" (UID: "6737f883-79c2-47be-b5a0-17ee7ab1f905"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:45:45.739523 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.739497    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6737f883-79c2-47be-b5a0-17ee7ab1f905-kube-api-access-pj7nr" (OuterVolumeSpecName: "kube-api-access-pj7nr") pod "6737f883-79c2-47be-b5a0-17ee7ab1f905" (UID: "6737f883-79c2-47be-b5a0-17ee7ab1f905"). InnerVolumeSpecName "kube-api-access-pj7nr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:45.739628 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.739612    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6737f883-79c2-47be-b5a0-17ee7ab1f905-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6737f883-79c2-47be-b5a0-17ee7ab1f905" (UID: "6737f883-79c2-47be-b5a0-17ee7ab1f905"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:45:45.739681 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.739632    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6737f883-79c2-47be-b5a0-17ee7ab1f905-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6737f883-79c2-47be-b5a0-17ee7ab1f905" (UID: "6737f883-79c2-47be-b5a0-17ee7ab1f905"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:45:45.839035 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.838961    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6737f883-79c2-47be-b5a0-17ee7ab1f905-maas-api-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:45.839035 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.838983    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6737f883-79c2-47be-b5a0-17ee7ab1f905-metrics-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:45.839035 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.838994    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj7nr\" (UniqueName: \"kubernetes.io/projected/6737f883-79c2-47be-b5a0-17ee7ab1f905-kube-api-access-pj7nr\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:45.839035 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.839003    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6737f883-79c2-47be-b5a0-17ee7ab1f905-openshift-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:45:45.959116 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.959080    2582 generic.go:356] "Generic (PLEG): container finished" podID="6737f883-79c2-47be-b5a0-17ee7ab1f905" containerID="ff66d6c978b8e4d7ace03522d1ab59da452a2ba469c6d3ba2efa9b62906d3c2c" exitCode=0
Sep 09 12:45:45.959290 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.959127    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"
Sep 09 12:45:45.959290 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.959156    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr" event={"ID":"6737f883-79c2-47be-b5a0-17ee7ab1f905","Type":"ContainerDied","Data":"ff66d6c978b8e4d7ace03522d1ab59da452a2ba469c6d3ba2efa9b62906d3c2c"}
Sep 09 12:45:45.959290 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.959205    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr" event={"ID":"6737f883-79c2-47be-b5a0-17ee7ab1f905","Type":"ContainerDied","Data":"ae7843e277479725bad25f8eb41fb933888d22e80baa0e3de5e8295a1df91862"}
Sep 09 12:45:45.959290 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.959226    2582 scope.go:117] "RemoveContainer" containerID="ff66d6c978b8e4d7ace03522d1ab59da452a2ba469c6d3ba2efa9b62906d3c2c"
Sep 09 12:45:45.969477 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.969460    2582 scope.go:117] "RemoveContainer" containerID="ff66d6c978b8e4d7ace03522d1ab59da452a2ba469c6d3ba2efa9b62906d3c2c"
Sep 09 12:45:45.969896 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:45:45.969871    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff66d6c978b8e4d7ace03522d1ab59da452a2ba469c6d3ba2efa9b62906d3c2c\": container with ID starting with ff66d6c978b8e4d7ace03522d1ab59da452a2ba469c6d3ba2efa9b62906d3c2c not found: ID does not exist" containerID="ff66d6c978b8e4d7ace03522d1ab59da452a2ba469c6d3ba2efa9b62906d3c2c"
Sep 09 12:45:45.970028 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.969912    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff66d6c978b8e4d7ace03522d1ab59da452a2ba469c6d3ba2efa9b62906d3c2c"} err="failed to get container status \"ff66d6c978b8e4d7ace03522d1ab59da452a2ba469c6d3ba2efa9b62906d3c2c\": rpc error: code = NotFound desc = could not find container \"ff66d6c978b8e4d7ace03522d1ab59da452a2ba469c6d3ba2efa9b62906d3c2c\": container with ID starting with ff66d6c978b8e4d7ace03522d1ab59da452a2ba469c6d3ba2efa9b62906d3c2c not found: ID does not exist"
Sep 09 12:45:45.980731 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.980710    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"]
Sep 09 12:45:45.984337 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:45.984318    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-590823-7c95855bf-j7ndr"]
Sep 09 12:45:46.824939 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:45:46.824906    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6737f883-79c2-47be-b5a0-17ee7ab1f905" path="/var/lib/kubelet/pods/6737f883-79c2-47be-b5a0-17ee7ab1f905/volumes"
Sep 09 12:46:02.398722 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:02.398683    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"]
Sep 09 12:46:04.222833 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.222801    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"]
Sep 09 12:46:04.223263 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223247    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerName="istio-proxy"
Sep 09 12:46:04.223263 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223265    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerName="istio-proxy"
Sep 09 12:46:04.223361 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223273    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea4982c8-fc35-4262-b114-97a17945e01d" containerName="maas-api"
Sep 09 12:46:04.223361 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223278    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea4982c8-fc35-4262-b114-97a17945e01d" containerName="maas-api"
Sep 09 12:46:04.223361 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223293    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38" containerName="maas-api"
Sep 09 12:46:04.223361 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223298    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38" containerName="maas-api"
Sep 09 12:46:04.223361 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223306    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6737f883-79c2-47be-b5a0-17ee7ab1f905" containerName="maas-api"
Sep 09 12:46:04.223361 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223311    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6737f883-79c2-47be-b5a0-17ee7ab1f905" containerName="maas-api"
Sep 09 12:46:04.223361 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223316    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8252e974-bf73-4cf0-9f87-46c3d4953046" containerName="istio-proxy"
Sep 09 12:46:04.223361 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223321    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8252e974-bf73-4cf0-9f87-46c3d4953046" containerName="istio-proxy"
Sep 09 12:46:04.223361 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223327    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d" containerName="main"
Sep 09 12:46:04.223361 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223331    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d" containerName="main"
Sep 09 12:46:04.223361 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223347    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8252e974-bf73-4cf0-9f87-46c3d4953046" containerName="istio-proxy"
Sep 09 12:46:04.223361 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223352    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8252e974-bf73-4cf0-9f87-46c3d4953046" containerName="istio-proxy"
Sep 09 12:46:04.223733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223410    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eca8bb0-6b2b-46fe-b7fa-dcfa0121a43d" containerName="main"
Sep 09 12:46:04.223733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223419    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8252e974-bf73-4cf0-9f87-46c3d4953046" containerName="istio-proxy"
Sep 09 12:46:04.223733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223424    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6737f883-79c2-47be-b5a0-17ee7ab1f905" containerName="maas-api"
Sep 09 12:46:04.223733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223430    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea4982c8-fc35-4262-b114-97a17945e01d" containerName="maas-api"
Sep 09 12:46:04.223733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223436    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerName="istio-proxy"
Sep 09 12:46:04.223733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223443    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerName="istio-proxy"
Sep 09 12:46:04.223733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.223450    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b97c50de-b01f-4b69-a1ba-ffd4a8ce5a38" containerName="maas-api"
Sep 09 12:46:04.228101 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.228083    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.230025 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.229994    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-610d01-openshift-default-dockercfg-9zczw\""
Sep 09 12:46:04.237827 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.237806    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"]
Sep 09 12:46:04.320326 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.320283    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7193f5e-78cd-47da-9a51-d579ba099780-istiod-ca-cert\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.320468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.320364    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkjp5\" (UniqueName: \"kubernetes.io/projected/b7193f5e-78cd-47da-9a51-d579ba099780-kube-api-access-jkjp5\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.320468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.320410    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-istio-envoy\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.320468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.320440    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-workload-certs\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.320576 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.320512    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7193f5e-78cd-47da-9a51-d579ba099780-istio-token\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.320576 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.320567    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-credential-socket\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.320640 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.320590    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7193f5e-78cd-47da-9a51-d579ba099780-istio-podinfo\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.320688 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.320665    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-workload-socket\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.320727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.320716    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-istio-data\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.421292 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.421263    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-istio-envoy\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.421456 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.421297    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-workload-certs\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.421456 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.421333    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7193f5e-78cd-47da-9a51-d579ba099780-istio-token\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.421456 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.421357    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-credential-socket\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.421456 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.421375    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7193f5e-78cd-47da-9a51-d579ba099780-istio-podinfo\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.421456 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.421405    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-workload-socket\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.421456 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.421437    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-istio-data\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.421769 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.421475    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7193f5e-78cd-47da-9a51-d579ba099780-istiod-ca-cert\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.421769 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.421522    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkjp5\" (UniqueName: \"kubernetes.io/projected/b7193f5e-78cd-47da-9a51-d579ba099780-kube-api-access-jkjp5\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.421769 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.421673    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-workload-certs\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.421769 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.421753    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-credential-socket\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.421972 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.421925    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-workload-socket\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.422079 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.422060    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-istio-data\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.422245 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.422228    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7193f5e-78cd-47da-9a51-d579ba099780-istiod-ca-cert\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.424294 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.424270    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-istio-envoy\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.424522 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.424501    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7193f5e-78cd-47da-9a51-d579ba099780-istio-podinfo\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.429086 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.429061    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7193f5e-78cd-47da-9a51-d579ba099780-istio-token\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.429305 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.429285    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkjp5\" (UniqueName: \"kubernetes.io/projected/b7193f5e-78cd-47da-9a51-d579ba099780-kube-api-access-jkjp5\") pod \"e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") " pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.539611 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.539588    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:04.818064 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.818036    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"]
Sep 09 12:46:04.819213 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:46:04.819174    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7193f5e_78cd_47da_9a51_d579ba099780.slice/crio-177a4fa0c910275cddc43edd71a70e3bfc3863f87b434153ca96cab99d2ec145 WatchSource:0}: Error finding container 177a4fa0c910275cddc43edd71a70e3bfc3863f87b434153ca96cab99d2ec145: Status 404 returned error can't find the container with id 177a4fa0c910275cddc43edd71a70e3bfc3863f87b434153ca96cab99d2ec145
Sep 09 12:46:04.821493 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.821455    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:46:04.821582 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.821541    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:46:04.821623 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:04.821578    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:46:05.077765 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:05.077681    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b" event={"ID":"b7193f5e-78cd-47da-9a51-d579ba099780","Type":"ContainerStarted","Data":"2ac3374065625618a5767884383a6b5572022fdb5d783e050e4858f41d01602b"}
Sep 09 12:46:05.077765 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:05.077744    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b" event={"ID":"b7193f5e-78cd-47da-9a51-d579ba099780","Type":"ContainerStarted","Data":"177a4fa0c910275cddc43edd71a70e3bfc3863f87b434153ca96cab99d2ec145"}
Sep 09 12:46:05.097867 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:05.097801    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b" podStartSLOduration=1.097787447 podStartE2EDuration="1.097787447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:46:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:46:05.095305519 +0000 UTC m=+1584.791870669" watchObservedRunningTime="2026-09-09 12:46:05.097787447 +0000 UTC m=+1584.794352597"
Sep 09 12:46:05.540602 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:05.540573    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:05.545297 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:05.545277    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:06.083912 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:06.083886    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:06.084703 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:06.084683    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:08.334695 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.334663    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"]
Sep 09 12:46:08.335151 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.335136    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerName="istio-proxy"
Sep 09 12:46:08.335151 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.335152    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e03a3bd-f94b-4c2e-9df6-7c23fac9a2fe" containerName="istio-proxy"
Sep 09 12:46:08.335241 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.335227    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8252e974-bf73-4cf0-9f87-46c3d4953046" containerName="istio-proxy"
Sep 09 12:46:08.338458 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.338438    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:08.340505 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.340484    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-610d01-serving-cert\""
Sep 09 12:46:08.348211 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.348061    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"]
Sep 09 12:46:08.461898 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.461832    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zr4k\" (UniqueName: \"kubernetes.io/projected/0af56795-b93a-4d84-b923-d3857d1083e1-kube-api-access-2zr4k\") pod \"maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:08.462063 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.461908    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0af56795-b93a-4d84-b923-d3857d1083e1-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:08.462063 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.461958    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af56795-b93a-4d84-b923-d3857d1083e1-maas-api-tls\") pod \"maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:08.462063 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.462036    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0af56795-b93a-4d84-b923-d3857d1083e1-metrics-certs\") pod \"maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:08.563542 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.563505    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0af56795-b93a-4d84-b923-d3857d1083e1-metrics-certs\") pod \"maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:08.563727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.563565    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zr4k\" (UniqueName: \"kubernetes.io/projected/0af56795-b93a-4d84-b923-d3857d1083e1-kube-api-access-2zr4k\") pod \"maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:08.563727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.563595    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0af56795-b93a-4d84-b923-d3857d1083e1-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:08.563727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.563652    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af56795-b93a-4d84-b923-d3857d1083e1-maas-api-tls\") pod \"maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:08.564257 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.564236    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0af56795-b93a-4d84-b923-d3857d1083e1-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:08.566180 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.566161    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af56795-b93a-4d84-b923-d3857d1083e1-maas-api-tls\") pod \"maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:08.566274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.566220    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0af56795-b93a-4d84-b923-d3857d1083e1-metrics-certs\") pod \"maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:08.571033 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.571012    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zr4k\" (UniqueName: \"kubernetes.io/projected/0af56795-b93a-4d84-b923-d3857d1083e1-kube-api-access-2zr4k\") pod \"maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:08.653118 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.653065    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:08.933504 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:08.933473    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"]
Sep 09 12:46:08.934882 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:46:08.934835    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af56795_b93a_4d84_b923_d3857d1083e1.slice/crio-a09c15bdbe80f50a0cb79fc4341f386ad90927683e2871d27c129392c3f34470 WatchSource:0}: Error finding container a09c15bdbe80f50a0cb79fc4341f386ad90927683e2871d27c129392c3f34470: Status 404 returned error can't find the container with id a09c15bdbe80f50a0cb79fc4341f386ad90927683e2871d27c129392c3f34470
Sep 09 12:46:09.102840 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:09.102806    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw" event={"ID":"0af56795-b93a-4d84-b923-d3857d1083e1","Type":"ContainerStarted","Data":"a09c15bdbe80f50a0cb79fc4341f386ad90927683e2871d27c129392c3f34470"}
Sep 09 12:46:10.110092 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:10.110054    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw" event={"ID":"0af56795-b93a-4d84-b923-d3857d1083e1","Type":"ContainerStarted","Data":"af0f073b66ad3b95d4fa06b7bb663d6514a94d2271ef0f23d556298938c8033b"}
Sep 09 12:46:10.110489 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:10.110112    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:10.128112 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:10.128058    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw" podStartSLOduration=1.6669549259999998 podStartE2EDuration="2.128042663s" totalImagesPullingTime="461.087737ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:46:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:46:10.126197018 +0000 UTC m=+1589.822762173" watchObservedRunningTime="2026-09-09 12:46:10.128042663 +0000 UTC m=+1589.824607813"
Sep 09 12:46:16.121212 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:16.121187    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:46:17.418955 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.418908    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh" podUID="411743cc-7116-437a-a6ce-c891d954d97a" containerName="main" containerID="cri-o://473181ee16ec6b9e15c03b27d4a027175f06072fcb56f0661c023001dd35c30c" gracePeriod=45
Sep 09 12:46:17.704117 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.704095    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:46:17.851447 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.851413    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-home\") pod \"411743cc-7116-437a-a6ce-c891d954d97a\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") "
Sep 09 12:46:17.851615 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.851461    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wngl4\" (UniqueName: \"kubernetes.io/projected/411743cc-7116-437a-a6ce-c891d954d97a-kube-api-access-wngl4\") pod \"411743cc-7116-437a-a6ce-c891d954d97a\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") "
Sep 09 12:46:17.851615 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.851499    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-model-cache\") pod \"411743cc-7116-437a-a6ce-c891d954d97a\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") "
Sep 09 12:46:17.851615 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.851554    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-dshm\") pod \"411743cc-7116-437a-a6ce-c891d954d97a\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") "
Sep 09 12:46:17.851615 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.851602    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/411743cc-7116-437a-a6ce-c891d954d97a-tls-certs\") pod \"411743cc-7116-437a-a6ce-c891d954d97a\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") "
Sep 09 12:46:17.851827 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.851662    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-home" (OuterVolumeSpecName: "home") pod "411743cc-7116-437a-a6ce-c891d954d97a" (UID: "411743cc-7116-437a-a6ce-c891d954d97a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:46:17.851827 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.851704    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-tmp-dir\") pod \"411743cc-7116-437a-a6ce-c891d954d97a\" (UID: \"411743cc-7116-437a-a6ce-c891d954d97a\") "
Sep 09 12:46:17.851827 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.851709    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-model-cache" (OuterVolumeSpecName: "model-cache") pod "411743cc-7116-437a-a6ce-c891d954d97a" (UID: "411743cc-7116-437a-a6ce-c891d954d97a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:46:17.852018 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.851948    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "411743cc-7116-437a-a6ce-c891d954d97a" (UID: "411743cc-7116-437a-a6ce-c891d954d97a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:46:17.852018 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.851995    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-home\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:46:17.852018 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.852011    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-model-cache\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:46:17.853717 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.853684    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-dshm" (OuterVolumeSpecName: "dshm") pod "411743cc-7116-437a-a6ce-c891d954d97a" (UID: "411743cc-7116-437a-a6ce-c891d954d97a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:46:17.853835 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.853748    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/411743cc-7116-437a-a6ce-c891d954d97a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "411743cc-7116-437a-a6ce-c891d954d97a" (UID: "411743cc-7116-437a-a6ce-c891d954d97a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:46:17.853835 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.853784    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/411743cc-7116-437a-a6ce-c891d954d97a-kube-api-access-wngl4" (OuterVolumeSpecName: "kube-api-access-wngl4") pod "411743cc-7116-437a-a6ce-c891d954d97a" (UID: "411743cc-7116-437a-a6ce-c891d954d97a"). InnerVolumeSpecName "kube-api-access-wngl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:46:17.953523 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.953433    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-tmp-dir\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:46:17.953675 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.953649    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wngl4\" (UniqueName: \"kubernetes.io/projected/411743cc-7116-437a-a6ce-c891d954d97a-kube-api-access-wngl4\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:46:17.953675 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.953666    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/411743cc-7116-437a-a6ce-c891d954d97a-dshm\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:46:17.953789 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:17.953683    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/411743cc-7116-437a-a6ce-c891d954d97a-tls-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:46:18.159788 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:18.159753    2582 generic.go:356] "Generic (PLEG): container finished" podID="411743cc-7116-437a-a6ce-c891d954d97a" containerID="473181ee16ec6b9e15c03b27d4a027175f06072fcb56f0661c023001dd35c30c" exitCode=0
Sep 09 12:46:18.159925 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:18.159802    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"
Sep 09 12:46:18.159925 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:18.159836    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh" event={"ID":"411743cc-7116-437a-a6ce-c891d954d97a","Type":"ContainerDied","Data":"473181ee16ec6b9e15c03b27d4a027175f06072fcb56f0661c023001dd35c30c"}
Sep 09 12:46:18.159925 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:18.159900    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh" event={"ID":"411743cc-7116-437a-a6ce-c891d954d97a","Type":"ContainerDied","Data":"822dbf6ed11a7facfc6c1dcd3a232e1899436fb618af9cb00dba73b9d0a5ed63"}
Sep 09 12:46:18.159925 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:18.159918    2582 scope.go:117] "RemoveContainer" containerID="473181ee16ec6b9e15c03b27d4a027175f06072fcb56f0661c023001dd35c30c"
Sep 09 12:46:18.169544 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:18.169528    2582 scope.go:117] "RemoveContainer" containerID="473181ee16ec6b9e15c03b27d4a027175f06072fcb56f0661c023001dd35c30c"
Sep 09 12:46:18.169879 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:46:18.169842    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"473181ee16ec6b9e15c03b27d4a027175f06072fcb56f0661c023001dd35c30c\": container with ID starting with 473181ee16ec6b9e15c03b27d4a027175f06072fcb56f0661c023001dd35c30c not found: ID does not exist" containerID="473181ee16ec6b9e15c03b27d4a027175f06072fcb56f0661c023001dd35c30c"
Sep 09 12:46:18.169951 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:18.169895    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"473181ee16ec6b9e15c03b27d4a027175f06072fcb56f0661c023001dd35c30c"} err="failed to get container status \"473181ee16ec6b9e15c03b27d4a027175f06072fcb56f0661c023001dd35c30c\": rpc error: code = NotFound desc = could not find container \"473181ee16ec6b9e15c03b27d4a027175f06072fcb56f0661c023001dd35c30c\": container with ID starting with 473181ee16ec6b9e15c03b27d4a027175f06072fcb56f0661c023001dd35c30c not found: ID does not exist"
Sep 09 12:46:18.182067 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:18.182041    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"]
Sep 09 12:46:18.184368 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:18.184339    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-283adb/rate-test-model-283adb-kserve-5b4df85c47-pgthh"]
Sep 09 12:46:18.826211 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:18.826178    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="411743cc-7116-437a-a6ce-c891d954d97a" path="/var/lib/kubelet/pods/411743cc-7116-437a-a6ce-c891d954d97a/volumes"
Sep 09 12:46:21.085290 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:21.085262    2582 patch_prober.go:28] interesting pod/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 09 12:46:21.085684 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:21.085324    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b" podUID="b7193f5e-78cd-47da-9a51-d579ba099780" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 09 12:46:21.181608 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:21.181582    2582 generic.go:356] "Generic (PLEG): container finished" podID="b7193f5e-78cd-47da-9a51-d579ba099780" containerID="2ac3374065625618a5767884383a6b5572022fdb5d783e050e4858f41d01602b" exitCode=0
Sep 09 12:46:21.181726 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:21.181658    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b" event={"ID":"b7193f5e-78cd-47da-9a51-d579ba099780","Type":"ContainerDied","Data":"2ac3374065625618a5767884383a6b5572022fdb5d783e050e4858f41d01602b"}
Sep 09 12:46:21.182090 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:21.182073    2582 scope.go:117] "RemoveContainer" containerID="2ac3374065625618a5767884383a6b5572022fdb5d783e050e4858f41d01602b"
Sep 09 12:46:21.183166 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:21.183104    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:46:21.183240 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:21.183172    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:46:21.183240 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:21.183204    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:46:22.195884 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:22.195356    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b" event={"ID":"b7193f5e-78cd-47da-9a51-d579ba099780","Type":"ContainerStarted","Data":"455e5760d531fafabe1ea740b44be53b96b42dcc482908cd50b3918cafcb1b33"}
Sep 09 12:46:22.540105 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:22.540069    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:22.540309 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:22.540242    2582 patch_prober.go:28] interesting pod/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 09 12:46:22.540393 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:22.540315    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b" podUID="b7193f5e-78cd-47da-9a51-d579ba099780" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 09 12:46:23.540488 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:23.540448    2582 patch_prober.go:28] interesting pod/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 09 12:46:23.540897 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:23.540519    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b" podUID="b7193f5e-78cd-47da-9a51-d579ba099780" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 09 12:46:24.540551 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:24.540512    2582 patch_prober.go:28] interesting pod/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 09 12:46:24.540982 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:24.540581    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b" podUID="b7193f5e-78cd-47da-9a51-d579ba099780" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 09 12:46:25.540124 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:25.540084    2582 patch_prober.go:28] interesting pod/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 09 12:46:25.540305 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:25.540147    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b" podUID="b7193f5e-78cd-47da-9a51-d579ba099780" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 09 12:46:26.539985 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:26.539948    2582 patch_prober.go:28] interesting pod/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 09 12:46:26.540382 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:26.540005    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b" podUID="b7193f5e-78cd-47da-9a51-d579ba099780" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 09 12:46:27.543819 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:27.543789    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:27.544304 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:27.544189    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:46:27.544917 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:46:27.544895    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:47:06.497392 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:06.497349    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hclrp_fec7ad7e-69c0-4f61-9c45-4214164224b5/payload-processing/0.log"
Sep 09 12:47:11.122583 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.122551    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"]
Sep 09 12:47:11.123257 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.123233    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="411743cc-7116-437a-a6ce-c891d954d97a" containerName="main"
Sep 09 12:47:11.123422 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.123409    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="411743cc-7116-437a-a6ce-c891d954d97a" containerName="main"
Sep 09 12:47:11.124051 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.124017    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="411743cc-7116-437a-a6ce-c891d954d97a" containerName="main"
Sep 09 12:47:11.127547 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.127523    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.130817 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.130795    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-610d01\"/\"openshift-service-ca.crt\""
Sep 09 12:47:11.131495 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.131473    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-610d01\"/\"kube-root-ca.crt\""
Sep 09 12:47:11.131620 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.131575    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-610d01\"/\"ipp-route-610d01-kserve-self-signed-certs\""
Sep 09 12:47:11.131620 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.131597    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-610d01\"/\"default-dockercfg-slgvt\""
Sep 09 12:47:11.137199 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.137179    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"]
Sep 09 12:47:11.226731 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.226696    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d46dw\" (UniqueName: \"kubernetes.io/projected/b282108d-ba10-48e8-b23e-de118dc5599d-kube-api-access-d46dw\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.226924 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.226779    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-home\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.226924 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.226822    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-tmp-dir\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.227035 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.227012    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-model-cache\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.227077 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.227051    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-dshm\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.227144 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.227124    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b282108d-ba10-48e8-b23e-de118dc5599d-tls-certs\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.328363 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.328329    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-model-cache\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.328545 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.328373    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-dshm\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.328545 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.328408    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b282108d-ba10-48e8-b23e-de118dc5599d-tls-certs\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.328545 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.328445    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d46dw\" (UniqueName: \"kubernetes.io/projected/b282108d-ba10-48e8-b23e-de118dc5599d-kube-api-access-d46dw\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.328545 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.328478    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-home\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.328545 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.328507    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-tmp-dir\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.328813 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.328758    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-model-cache\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.328908 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.328885    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-home\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.328961 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.328906    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-tmp-dir\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.330926 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.330903    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-dshm\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.331201 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.331179    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b282108d-ba10-48e8-b23e-de118dc5599d-tls-certs\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.338541 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.338522    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d46dw\" (UniqueName: \"kubernetes.io/projected/b282108d-ba10-48e8-b23e-de118dc5599d-kube-api-access-d46dw\") pod \"ipp-route-610d01-kserve-764687fc4c-sd8jt\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") " pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.440294 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.440196    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:11.716946 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:11.716923    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"]
Sep 09 12:47:11.718417 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:47:11.718395    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb282108d_ba10_48e8_b23e_de118dc5599d.slice/crio-83f561aed8768746680b768cebce6a8a5f4a30978a8ec8b15470cb1d7c23a7db WatchSource:0}: Error finding container 83f561aed8768746680b768cebce6a8a5f4a30978a8ec8b15470cb1d7c23a7db: Status 404 returned error can't find the container with id 83f561aed8768746680b768cebce6a8a5f4a30978a8ec8b15470cb1d7c23a7db
Sep 09 12:47:12.500278 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:12.500242    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt" event={"ID":"b282108d-ba10-48e8-b23e-de118dc5599d","Type":"ContainerStarted","Data":"1a78d6359a4fc2dc94c3719b1da8559c296dd050e8e5cd72889339571cc159be"}
Sep 09 12:47:12.500709 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:12.500294    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt" event={"ID":"b282108d-ba10-48e8-b23e-de118dc5599d","Type":"ContainerStarted","Data":"83f561aed8768746680b768cebce6a8a5f4a30978a8ec8b15470cb1d7c23a7db"}
Sep 09 12:47:12.517427 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:12.517374    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt" podStartSLOduration=1.51736027 podStartE2EDuration="1.51736027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:47:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:47:12.515372488 +0000 UTC m=+1652.211937653" watchObservedRunningTime="2026-09-09 12:47:12.51736027 +0000 UTC m=+1652.213925422"
Sep 09 12:47:21.440719 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:21.440683    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:21.441105 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:21.440755    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:21.453075 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:21.453053    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:21.567063 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:21.567041    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:47:34.284940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:34.284906    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hclrp_fec7ad7e-69c0-4f61-9c45-4214164224b5/payload-processing/0.log"
Sep 09 12:47:35.029221 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.029192    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"]
Sep 09 12:47:35.033005 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.032982    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.035142 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.035121    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-90dee1-openshift-default-dockercfg-kx9wd\""
Sep 09 12:47:35.047018 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.046998    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"]
Sep 09 12:47:35.156538 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.156504    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ksxz\" (UniqueName: \"kubernetes.io/projected/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-kube-api-access-9ksxz\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.156730 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.156550    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-token\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.156730 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.156609    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-credential-socket\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.156730 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.156627    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-podinfo\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.156730 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.156647    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-data\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.156730 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.156668    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-workload-socket\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.156730 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.156682    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-envoy\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.157051 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.156749    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-workload-certs\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.157051 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.156796    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.257923 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.257878    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-credential-socket\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.257923 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.257927    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-podinfo\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.258170 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.257949    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-data\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.258170 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.257973    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-workload-socket\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.258170 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.257989    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-envoy\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.258170 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.258016    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-workload-certs\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.258170 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.258037    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.258170 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.258133    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ksxz\" (UniqueName: \"kubernetes.io/projected/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-kube-api-access-9ksxz\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.258170 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.258159    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-token\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.258547 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.258351    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-credential-socket\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.258601 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.258570    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-data\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.258655 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.258601    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-workload-socket\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.258714 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.258670    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-workload-certs\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.259232 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.259204    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.260515 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.260497    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-envoy\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.260639 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.260621    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-podinfo\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.266049 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.266028    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-token\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.266260 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.266242    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ksxz\" (UniqueName: \"kubernetes.io/projected/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-kube-api-access-9ksxz\") pod \"e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") " pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.346143 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.346060    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:35.625453 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.625426    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"]
Sep 09 12:47:35.627393 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:47:35.627366    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda47c6ab0_5032_4030_afaa_72a2d9d90c7f.slice/crio-eebe94a4ab88ce5fca77009fa7d99157044de068d48406b7373e74484b7a8b87 WatchSource:0}: Error finding container eebe94a4ab88ce5fca77009fa7d99157044de068d48406b7373e74484b7a8b87: Status 404 returned error can't find the container with id eebe94a4ab88ce5fca77009fa7d99157044de068d48406b7373e74484b7a8b87
Sep 09 12:47:35.629749 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.629705    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:47:35.629877 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.629796    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:47:35.629962 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.629893    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:47:35.641072 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:35.641048    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9" event={"ID":"a47c6ab0-5032-4030-afaa-72a2d9d90c7f","Type":"ContainerStarted","Data":"eebe94a4ab88ce5fca77009fa7d99157044de068d48406b7373e74484b7a8b87"}
Sep 09 12:47:36.652495 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:36.652461    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9" event={"ID":"a47c6ab0-5032-4030-afaa-72a2d9d90c7f","Type":"ContainerStarted","Data":"37313c7010f5645ddc93cb1a61257ae2ae31635975ae441737e5bc74308a7f15"}
Sep 09 12:47:36.673899 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:36.673823    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9" podStartSLOduration=1.6738074790000002 podStartE2EDuration="1.673807479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:47:36.671517726 +0000 UTC m=+1676.368082877" watchObservedRunningTime="2026-09-09 12:47:36.673807479 +0000 UTC m=+1676.370372700"
Sep 09 12:47:37.347196 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:37.347169    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:37.352708 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:37.352683    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:37.658923 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:37.658817    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:37.659830 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:37.659809    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:57.789128 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:57.789095    2582 generic.go:356] "Generic (PLEG): container finished" podID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerID="37313c7010f5645ddc93cb1a61257ae2ae31635975ae441737e5bc74308a7f15" exitCode=0
Sep 09 12:47:57.789569 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:57.789179    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9" event={"ID":"a47c6ab0-5032-4030-afaa-72a2d9d90c7f","Type":"ContainerDied","Data":"37313c7010f5645ddc93cb1a61257ae2ae31635975ae441737e5bc74308a7f15"}
Sep 09 12:47:57.789665 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:57.789650    2582 scope.go:117] "RemoveContainer" containerID="37313c7010f5645ddc93cb1a61257ae2ae31635975ae441737e5bc74308a7f15"
Sep 09 12:47:57.790928 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:57.790901    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:47:57.791014 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:57.790968    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:47:57.791014 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:57.790998    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 09 12:47:58.797862 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:58.797828    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9" event={"ID":"a47c6ab0-5032-4030-afaa-72a2d9d90c7f","Type":"ContainerStarted","Data":"b6297eee902ba7a2a1120b5e2dbfa589e5c7917e2cde1eb0e78a603268e02738"}
Sep 09 12:47:59.346418 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:59.346388    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:47:59.346662 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:59.346640    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 09 12:47:59.346733 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:47:59.346697    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9" podUID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 09 12:48:00.346519 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:00.346483    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 09 12:48:00.346956 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:00.346538    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9" podUID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 09 12:48:01.346783 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:01.346751    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 09 12:48:01.347190 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:01.346801    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9" podUID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 09 12:48:02.347117 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:02.347030    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 09 12:48:02.347557 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:02.347115    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9" podUID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 09 12:48:03.346814 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:03.346775    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 09 12:48:03.347019 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:03.346837    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9" podUID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 09 12:48:04.346808 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:04.346771    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 09 12:48:04.347254 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:04.346826    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9" podUID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 09 12:48:05.350698 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:05.350664    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:48:05.351113 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:05.351054    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:48:05.351782 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:05.351765    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:48:17.672564 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:17.672522    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"]
Sep 09 12:48:17.673064 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:17.672767    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9" podUID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerName="istio-proxy" containerID="cri-o://b6297eee902ba7a2a1120b5e2dbfa589e5c7917e2cde1eb0e78a603268e02738" gracePeriod=30
Sep 09 12:48:20.353794 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:20.353747    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 12:48:20.354191 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:20.353826    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9" podUID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 12:48:22.923612 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:22.923587    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:48:22.952255 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:22.952187    2582 generic.go:356] "Generic (PLEG): container finished" podID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerID="b6297eee902ba7a2a1120b5e2dbfa589e5c7917e2cde1eb0e78a603268e02738" exitCode=0
Sep 09 12:48:22.952255 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:22.952239    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"
Sep 09 12:48:22.952449 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:22.952352    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9" event={"ID":"a47c6ab0-5032-4030-afaa-72a2d9d90c7f","Type":"ContainerDied","Data":"b6297eee902ba7a2a1120b5e2dbfa589e5c7917e2cde1eb0e78a603268e02738"}
Sep 09 12:48:22.952449 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:22.952390    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9" event={"ID":"a47c6ab0-5032-4030-afaa-72a2d9d90c7f","Type":"ContainerDied","Data":"eebe94a4ab88ce5fca77009fa7d99157044de068d48406b7373e74484b7a8b87"}
Sep 09 12:48:22.952449 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:22.952415    2582 scope.go:117] "RemoveContainer" containerID="b6297eee902ba7a2a1120b5e2dbfa589e5c7917e2cde1eb0e78a603268e02738"
Sep 09 12:48:22.962010 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:22.961993    2582 scope.go:117] "RemoveContainer" containerID="37313c7010f5645ddc93cb1a61257ae2ae31635975ae441737e5bc74308a7f15"
Sep 09 12:48:22.970549 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:22.970530    2582 scope.go:117] "RemoveContainer" containerID="b6297eee902ba7a2a1120b5e2dbfa589e5c7917e2cde1eb0e78a603268e02738"
Sep 09 12:48:22.970838 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:48:22.970818    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6297eee902ba7a2a1120b5e2dbfa589e5c7917e2cde1eb0e78a603268e02738\": container with ID starting with b6297eee902ba7a2a1120b5e2dbfa589e5c7917e2cde1eb0e78a603268e02738 not found: ID does not exist" containerID="b6297eee902ba7a2a1120b5e2dbfa589e5c7917e2cde1eb0e78a603268e02738"
Sep 09 12:48:22.970923 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:22.970870    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6297eee902ba7a2a1120b5e2dbfa589e5c7917e2cde1eb0e78a603268e02738"} err="failed to get container status \"b6297eee902ba7a2a1120b5e2dbfa589e5c7917e2cde1eb0e78a603268e02738\": rpc error: code = NotFound desc = could not find container \"b6297eee902ba7a2a1120b5e2dbfa589e5c7917e2cde1eb0e78a603268e02738\": container with ID starting with b6297eee902ba7a2a1120b5e2dbfa589e5c7917e2cde1eb0e78a603268e02738 not found: ID does not exist"
Sep 09 12:48:22.970923 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:22.970891    2582 scope.go:117] "RemoveContainer" containerID="37313c7010f5645ddc93cb1a61257ae2ae31635975ae441737e5bc74308a7f15"
Sep 09 12:48:22.971218 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:48:22.971179    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37313c7010f5645ddc93cb1a61257ae2ae31635975ae441737e5bc74308a7f15\": container with ID starting with 37313c7010f5645ddc93cb1a61257ae2ae31635975ae441737e5bc74308a7f15 not found: ID does not exist" containerID="37313c7010f5645ddc93cb1a61257ae2ae31635975ae441737e5bc74308a7f15"
Sep 09 12:48:22.971273 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:22.971229    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37313c7010f5645ddc93cb1a61257ae2ae31635975ae441737e5bc74308a7f15"} err="failed to get container status \"37313c7010f5645ddc93cb1a61257ae2ae31635975ae441737e5bc74308a7f15\": rpc error: code = NotFound desc = could not find container \"37313c7010f5645ddc93cb1a61257ae2ae31635975ae441737e5bc74308a7f15\": container with ID starting with 37313c7010f5645ddc93cb1a61257ae2ae31635975ae441737e5bc74308a7f15 not found: ID does not exist"
Sep 09 12:48:23.015316 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.015288    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-token\") pod \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") "
Sep 09 12:48:23.015423 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.015329    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-credential-socket\") pod \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") "
Sep 09 12:48:23.015423 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.015362    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-workload-certs\") pod \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") "
Sep 09 12:48:23.015423 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.015395    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ksxz\" (UniqueName: \"kubernetes.io/projected/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-kube-api-access-9ksxz\") pod \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") "
Sep 09 12:48:23.015589 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.015440    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-workload-socket\") pod \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") "
Sep 09 12:48:23.015589 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.015494    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istiod-ca-cert\") pod \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") "
Sep 09 12:48:23.015589 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.015528    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-podinfo\") pod \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") "
Sep 09 12:48:23.015589 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.015567    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-envoy\") pod \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") "
Sep 09 12:48:23.015782 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.015681    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-data\") pod \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\" (UID: \"a47c6ab0-5032-4030-afaa-72a2d9d90c7f\") "
Sep 09 12:48:23.015782 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.015707    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a47c6ab0-5032-4030-afaa-72a2d9d90c7f" (UID: "a47c6ab0-5032-4030-afaa-72a2d9d90c7f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:48:23.015782 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.015713    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a47c6ab0-5032-4030-afaa-72a2d9d90c7f" (UID: "a47c6ab0-5032-4030-afaa-72a2d9d90c7f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:48:23.015782 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.015743    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a47c6ab0-5032-4030-afaa-72a2d9d90c7f" (UID: "a47c6ab0-5032-4030-afaa-72a2d9d90c7f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:48:23.016259 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.016044    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-data" (OuterVolumeSpecName: "istio-data") pod "a47c6ab0-5032-4030-afaa-72a2d9d90c7f" (UID: "a47c6ab0-5032-4030-afaa-72a2d9d90c7f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:48:23.016259 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.016123    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a47c6ab0-5032-4030-afaa-72a2d9d90c7f" (UID: "a47c6ab0-5032-4030-afaa-72a2d9d90c7f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:48:23.016388 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.016292    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:23.016388 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.016315    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-credential-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:23.016388 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.016330    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-workload-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:23.016388 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.016345    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-workload-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:23.016388 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.016359    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istiod-ca-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:23.017615 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.017593    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-token" (OuterVolumeSpecName: "istio-token") pod "a47c6ab0-5032-4030-afaa-72a2d9d90c7f" (UID: "a47c6ab0-5032-4030-afaa-72a2d9d90c7f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:48:23.017760 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.017740    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a47c6ab0-5032-4030-afaa-72a2d9d90c7f" (UID: "a47c6ab0-5032-4030-afaa-72a2d9d90c7f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:48:23.017841 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.017781    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-kube-api-access-9ksxz" (OuterVolumeSpecName: "kube-api-access-9ksxz") pod "a47c6ab0-5032-4030-afaa-72a2d9d90c7f" (UID: "a47c6ab0-5032-4030-afaa-72a2d9d90c7f"). InnerVolumeSpecName "kube-api-access-9ksxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:48:23.018001 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.017985    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a47c6ab0-5032-4030-afaa-72a2d9d90c7f" (UID: "a47c6ab0-5032-4030-afaa-72a2d9d90c7f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:48:23.117760 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.117732    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-podinfo\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:23.117760 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.117755    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-envoy\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:23.117760 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.117764    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-istio-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:23.118002 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.117772    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ksxz\" (UniqueName: \"kubernetes.io/projected/a47c6ab0-5032-4030-afaa-72a2d9d90c7f-kube-api-access-9ksxz\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:23.274212 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.274186    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"]
Sep 09 12:48:23.277840 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:23.277817    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-90dee1-openshift-default-68d6b75fb8-9t6q9"]
Sep 09 12:48:24.825127 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:24.825093    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" path="/var/lib/kubelet/pods/a47c6ab0-5032-4030-afaa-72a2d9d90c7f/volumes"
Sep 09 12:48:36.883554 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:36.883517    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"]
Sep 09 12:48:36.884174 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:36.883766    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw" podUID="0af56795-b93a-4d84-b923-d3857d1083e1" containerName="maas-api" containerID="cri-o://af0f073b66ad3b95d4fa06b7bb663d6514a94d2271ef0f23d556298938c8033b" gracePeriod=30
Sep 09 12:48:37.048806 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:37.048768    2582 generic.go:356] "Generic (PLEG): container finished" podID="0af56795-b93a-4d84-b923-d3857d1083e1" containerID="af0f073b66ad3b95d4fa06b7bb663d6514a94d2271ef0f23d556298938c8033b" exitCode=0
Sep 09 12:48:37.048996 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:37.048842    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw" event={"ID":"0af56795-b93a-4d84-b923-d3857d1083e1","Type":"ContainerDied","Data":"af0f073b66ad3b95d4fa06b7bb663d6514a94d2271ef0f23d556298938c8033b"}
Sep 09 12:48:37.153339 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:37.153317    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:48:37.240449 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:37.240414    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0af56795-b93a-4d84-b923-d3857d1083e1-metrics-certs\") pod \"0af56795-b93a-4d84-b923-d3857d1083e1\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") "
Sep 09 12:48:37.240628 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:37.240462    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zr4k\" (UniqueName: \"kubernetes.io/projected/0af56795-b93a-4d84-b923-d3857d1083e1-kube-api-access-2zr4k\") pod \"0af56795-b93a-4d84-b923-d3857d1083e1\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") "
Sep 09 12:48:37.240628 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:37.240501    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0af56795-b93a-4d84-b923-d3857d1083e1-openshift-service-ca\") pod \"0af56795-b93a-4d84-b923-d3857d1083e1\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") "
Sep 09 12:48:37.240628 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:37.240537    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af56795-b93a-4d84-b923-d3857d1083e1-maas-api-tls\") pod \"0af56795-b93a-4d84-b923-d3857d1083e1\" (UID: \"0af56795-b93a-4d84-b923-d3857d1083e1\") "
Sep 09 12:48:37.240838 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:37.240815    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0af56795-b93a-4d84-b923-d3857d1083e1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0af56795-b93a-4d84-b923-d3857d1083e1" (UID: "0af56795-b93a-4d84-b923-d3857d1083e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:48:37.241011 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:37.240991    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0af56795-b93a-4d84-b923-d3857d1083e1-openshift-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:37.242762 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:37.242734    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0af56795-b93a-4d84-b923-d3857d1083e1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0af56795-b93a-4d84-b923-d3857d1083e1" (UID: "0af56795-b93a-4d84-b923-d3857d1083e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:48:37.242899 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:37.242758    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0af56795-b93a-4d84-b923-d3857d1083e1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0af56795-b93a-4d84-b923-d3857d1083e1" (UID: "0af56795-b93a-4d84-b923-d3857d1083e1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:48:37.242899 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:37.242762    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0af56795-b93a-4d84-b923-d3857d1083e1-kube-api-access-2zr4k" (OuterVolumeSpecName: "kube-api-access-2zr4k") pod "0af56795-b93a-4d84-b923-d3857d1083e1" (UID: "0af56795-b93a-4d84-b923-d3857d1083e1"). InnerVolumeSpecName "kube-api-access-2zr4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:48:37.341942 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:37.341906    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zr4k\" (UniqueName: \"kubernetes.io/projected/0af56795-b93a-4d84-b923-d3857d1083e1-kube-api-access-2zr4k\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:37.341942 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:37.341939    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af56795-b93a-4d84-b923-d3857d1083e1-maas-api-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:37.342116 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:37.341951    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0af56795-b93a-4d84-b923-d3857d1083e1-metrics-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:38.056511 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:38.056482    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"
Sep 09 12:48:38.056967 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:38.056519    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw" event={"ID":"0af56795-b93a-4d84-b923-d3857d1083e1","Type":"ContainerDied","Data":"a09c15bdbe80f50a0cb79fc4341f386ad90927683e2871d27c129392c3f34470"}
Sep 09 12:48:38.056967 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:38.056580    2582 scope.go:117] "RemoveContainer" containerID="af0f073b66ad3b95d4fa06b7bb663d6514a94d2271ef0f23d556298938c8033b"
Sep 09 12:48:38.081267 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:38.081245    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"]
Sep 09 12:48:38.085518 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:38.085495    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-610d01-7fb8fc8f85-nznjw"]
Sep 09 12:48:38.824721 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:38.824693    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0af56795-b93a-4d84-b923-d3857d1083e1" path="/var/lib/kubelet/pods/0af56795-b93a-4d84-b923-d3857d1083e1/volumes"
Sep 09 12:48:42.493933 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:42.493890    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"]
Sep 09 12:48:57.514867 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.514806    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt" podUID="b282108d-ba10-48e8-b23e-de118dc5599d" containerName="main" containerID="cri-o://1a78d6359a4fc2dc94c3719b1da8559c296dd050e8e5cd72889339571cc159be" gracePeriod=45
Sep 09 12:48:57.771235 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.771182    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:48:57.825772 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.825744    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b282108d-ba10-48e8-b23e-de118dc5599d-tls-certs\") pod \"b282108d-ba10-48e8-b23e-de118dc5599d\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") "
Sep 09 12:48:57.825971 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.825787    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d46dw\" (UniqueName: \"kubernetes.io/projected/b282108d-ba10-48e8-b23e-de118dc5599d-kube-api-access-d46dw\") pod \"b282108d-ba10-48e8-b23e-de118dc5599d\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") "
Sep 09 12:48:57.825971 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.825889    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-home\") pod \"b282108d-ba10-48e8-b23e-de118dc5599d\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") "
Sep 09 12:48:57.825971 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.825943    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-dshm\") pod \"b282108d-ba10-48e8-b23e-de118dc5599d\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") "
Sep 09 12:48:57.826146 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.826032    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-model-cache\") pod \"b282108d-ba10-48e8-b23e-de118dc5599d\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") "
Sep 09 12:48:57.826146 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.826056    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-tmp-dir\") pod \"b282108d-ba10-48e8-b23e-de118dc5599d\" (UID: \"b282108d-ba10-48e8-b23e-de118dc5599d\") "
Sep 09 12:48:57.826320 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.826177    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-home" (OuterVolumeSpecName: "home") pod "b282108d-ba10-48e8-b23e-de118dc5599d" (UID: "b282108d-ba10-48e8-b23e-de118dc5599d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:48:57.826447 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.826398    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b282108d-ba10-48e8-b23e-de118dc5599d" (UID: "b282108d-ba10-48e8-b23e-de118dc5599d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:48:57.826737 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.826477    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-home\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:57.826737 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.826484    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-model-cache" (OuterVolumeSpecName: "model-cache") pod "b282108d-ba10-48e8-b23e-de118dc5599d" (UID: "b282108d-ba10-48e8-b23e-de118dc5599d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:48:57.828431 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.828383    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b282108d-ba10-48e8-b23e-de118dc5599d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b282108d-ba10-48e8-b23e-de118dc5599d" (UID: "b282108d-ba10-48e8-b23e-de118dc5599d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:48:57.828543 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.828457    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-dshm" (OuterVolumeSpecName: "dshm") pod "b282108d-ba10-48e8-b23e-de118dc5599d" (UID: "b282108d-ba10-48e8-b23e-de118dc5599d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:48:57.828651 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.828626    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b282108d-ba10-48e8-b23e-de118dc5599d-kube-api-access-d46dw" (OuterVolumeSpecName: "kube-api-access-d46dw") pod "b282108d-ba10-48e8-b23e-de118dc5599d" (UID: "b282108d-ba10-48e8-b23e-de118dc5599d"). InnerVolumeSpecName "kube-api-access-d46dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:48:57.927991 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.927967    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-dshm\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:57.927991 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.927988    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-model-cache\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:57.927991 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.927998    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b282108d-ba10-48e8-b23e-de118dc5599d-tmp-dir\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:57.928179 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.928005    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b282108d-ba10-48e8-b23e-de118dc5599d-tls-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:57.928179 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:57.928013    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d46dw\" (UniqueName: \"kubernetes.io/projected/b282108d-ba10-48e8-b23e-de118dc5599d-kube-api-access-d46dw\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:48:58.184274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:58.184186    2582 generic.go:356] "Generic (PLEG): container finished" podID="b282108d-ba10-48e8-b23e-de118dc5599d" containerID="1a78d6359a4fc2dc94c3719b1da8559c296dd050e8e5cd72889339571cc159be" exitCode=0
Sep 09 12:48:58.184274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:58.184229    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt" event={"ID":"b282108d-ba10-48e8-b23e-de118dc5599d","Type":"ContainerDied","Data":"1a78d6359a4fc2dc94c3719b1da8559c296dd050e8e5cd72889339571cc159be"}
Sep 09 12:48:58.184274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:58.184266    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt" event={"ID":"b282108d-ba10-48e8-b23e-de118dc5599d","Type":"ContainerDied","Data":"83f561aed8768746680b768cebce6a8a5f4a30978a8ec8b15470cb1d7c23a7db"}
Sep 09 12:48:58.184544 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:58.184289    2582 scope.go:117] "RemoveContainer" containerID="1a78d6359a4fc2dc94c3719b1da8559c296dd050e8e5cd72889339571cc159be"
Sep 09 12:48:58.184544 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:58.184232    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"
Sep 09 12:48:58.193809 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:58.193794    2582 scope.go:117] "RemoveContainer" containerID="1a78d6359a4fc2dc94c3719b1da8559c296dd050e8e5cd72889339571cc159be"
Sep 09 12:48:58.194114 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:48:58.194096    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a78d6359a4fc2dc94c3719b1da8559c296dd050e8e5cd72889339571cc159be\": container with ID starting with 1a78d6359a4fc2dc94c3719b1da8559c296dd050e8e5cd72889339571cc159be not found: ID does not exist" containerID="1a78d6359a4fc2dc94c3719b1da8559c296dd050e8e5cd72889339571cc159be"
Sep 09 12:48:58.194185 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:58.194126    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a78d6359a4fc2dc94c3719b1da8559c296dd050e8e5cd72889339571cc159be"} err="failed to get container status \"1a78d6359a4fc2dc94c3719b1da8559c296dd050e8e5cd72889339571cc159be\": rpc error: code = NotFound desc = could not find container \"1a78d6359a4fc2dc94c3719b1da8559c296dd050e8e5cd72889339571cc159be\": container with ID starting with 1a78d6359a4fc2dc94c3719b1da8559c296dd050e8e5cd72889339571cc159be not found: ID does not exist"
Sep 09 12:48:58.206704 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:58.206682    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"]
Sep 09 12:48:58.209091 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:58.209070    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-610d01/ipp-route-610d01-kserve-764687fc4c-sd8jt"]
Sep 09 12:48:58.825065 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:48:58.825026    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b282108d-ba10-48e8-b23e-de118dc5599d" path="/var/lib/kubelet/pods/b282108d-ba10-48e8-b23e-de118dc5599d/volumes"
Sep 09 12:49:26.300283 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:26.300234    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"]
Sep 09 12:49:26.301187 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:26.301152    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b" podUID="b7193f5e-78cd-47da-9a51-d579ba099780" containerName="istio-proxy" containerID="cri-o://455e5760d531fafabe1ea740b44be53b96b42dcc482908cd50b3918cafcb1b33" gracePeriod=30
Sep 09 12:49:27.546444 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:27.546408    2582 patch_prober.go:28] interesting pod/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 12:49:27.546831 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:27.546467    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b" podUID="b7193f5e-78cd-47da-9a51-d579ba099780" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 12:49:31.575922 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.575840    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:49:31.727360 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.727328    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-workload-socket\") pod \"b7193f5e-78cd-47da-9a51-d579ba099780\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") "
Sep 09 12:49:31.727360 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.727366    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-istio-data\") pod \"b7193f5e-78cd-47da-9a51-d579ba099780\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") "
Sep 09 12:49:31.727609 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.727408    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-istio-envoy\") pod \"b7193f5e-78cd-47da-9a51-d579ba099780\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") "
Sep 09 12:49:31.727609 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.727446    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkjp5\" (UniqueName: \"kubernetes.io/projected/b7193f5e-78cd-47da-9a51-d579ba099780-kube-api-access-jkjp5\") pod \"b7193f5e-78cd-47da-9a51-d579ba099780\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") "
Sep 09 12:49:31.727609 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.727483    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7193f5e-78cd-47da-9a51-d579ba099780-istio-podinfo\") pod \"b7193f5e-78cd-47da-9a51-d579ba099780\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") "
Sep 09 12:49:31.727609 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.727515    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-workload-certs\") pod \"b7193f5e-78cd-47da-9a51-d579ba099780\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") "
Sep 09 12:49:31.727609 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.727579    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7193f5e-78cd-47da-9a51-d579ba099780-istio-token\") pod \"b7193f5e-78cd-47da-9a51-d579ba099780\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") "
Sep 09 12:49:31.727907 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.727646    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b7193f5e-78cd-47da-9a51-d579ba099780" (UID: "b7193f5e-78cd-47da-9a51-d579ba099780"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:49:31.727907 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.727655    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7193f5e-78cd-47da-9a51-d579ba099780-istiod-ca-cert\") pod \"b7193f5e-78cd-47da-9a51-d579ba099780\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") "
Sep 09 12:49:31.727907 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.727683    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-credential-socket\") pod \"b7193f5e-78cd-47da-9a51-d579ba099780\" (UID: \"b7193f5e-78cd-47da-9a51-d579ba099780\") "
Sep 09 12:49:31.727907 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.727690    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-istio-data" (OuterVolumeSpecName: "istio-data") pod "b7193f5e-78cd-47da-9a51-d579ba099780" (UID: "b7193f5e-78cd-47da-9a51-d579ba099780"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:49:31.727907 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.727839    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b7193f5e-78cd-47da-9a51-d579ba099780" (UID: "b7193f5e-78cd-47da-9a51-d579ba099780"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:49:31.728195 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.728070    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-workload-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:49:31.728195 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.728093    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-workload-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:49:31.728195 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.728108    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-istio-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:49:31.728195 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.728071    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b7193f5e-78cd-47da-9a51-d579ba099780" (UID: "b7193f5e-78cd-47da-9a51-d579ba099780"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:49:31.728195 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.728077    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7193f5e-78cd-47da-9a51-d579ba099780-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b7193f5e-78cd-47da-9a51-d579ba099780" (UID: "b7193f5e-78cd-47da-9a51-d579ba099780"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:49:31.729947 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.729915    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7193f5e-78cd-47da-9a51-d579ba099780-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b7193f5e-78cd-47da-9a51-d579ba099780" (UID: "b7193f5e-78cd-47da-9a51-d579ba099780"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:49:31.730116 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.730094    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7193f5e-78cd-47da-9a51-d579ba099780-kube-api-access-jkjp5" (OuterVolumeSpecName: "kube-api-access-jkjp5") pod "b7193f5e-78cd-47da-9a51-d579ba099780" (UID: "b7193f5e-78cd-47da-9a51-d579ba099780"). InnerVolumeSpecName "kube-api-access-jkjp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:49:31.730311 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.730291    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7193f5e-78cd-47da-9a51-d579ba099780-istio-token" (OuterVolumeSpecName: "istio-token") pod "b7193f5e-78cd-47da-9a51-d579ba099780" (UID: "b7193f5e-78cd-47da-9a51-d579ba099780"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:49:31.730354 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.730309    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b7193f5e-78cd-47da-9a51-d579ba099780" (UID: "b7193f5e-78cd-47da-9a51-d579ba099780"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:49:31.828940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.828811    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7193f5e-78cd-47da-9a51-d579ba099780-istio-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:49:31.828940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.828844    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7193f5e-78cd-47da-9a51-d579ba099780-istiod-ca-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:49:31.828940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.828886    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-credential-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:49:31.828940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.828898    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7193f5e-78cd-47da-9a51-d579ba099780-istio-envoy\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:49:31.828940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.828909    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkjp5\" (UniqueName: \"kubernetes.io/projected/b7193f5e-78cd-47da-9a51-d579ba099780-kube-api-access-jkjp5\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:49:31.828940 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:31.828920    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7193f5e-78cd-47da-9a51-d579ba099780-istio-podinfo\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:49:32.387291 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:32.387251    2582 generic.go:356] "Generic (PLEG): container finished" podID="b7193f5e-78cd-47da-9a51-d579ba099780" containerID="455e5760d531fafabe1ea740b44be53b96b42dcc482908cd50b3918cafcb1b33" exitCode=0
Sep 09 12:49:32.387499 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:32.387305    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"
Sep 09 12:49:32.387499 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:32.387343    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b" event={"ID":"b7193f5e-78cd-47da-9a51-d579ba099780","Type":"ContainerDied","Data":"455e5760d531fafabe1ea740b44be53b96b42dcc482908cd50b3918cafcb1b33"}
Sep 09 12:49:32.387499 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:32.387413    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b" event={"ID":"b7193f5e-78cd-47da-9a51-d579ba099780","Type":"ContainerDied","Data":"177a4fa0c910275cddc43edd71a70e3bfc3863f87b434153ca96cab99d2ec145"}
Sep 09 12:49:32.387499 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:32.387443    2582 scope.go:117] "RemoveContainer" containerID="455e5760d531fafabe1ea740b44be53b96b42dcc482908cd50b3918cafcb1b33"
Sep 09 12:49:32.397623 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:32.397604    2582 scope.go:117] "RemoveContainer" containerID="2ac3374065625618a5767884383a6b5572022fdb5d783e050e4858f41d01602b"
Sep 09 12:49:32.406709 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:32.406689    2582 scope.go:117] "RemoveContainer" containerID="455e5760d531fafabe1ea740b44be53b96b42dcc482908cd50b3918cafcb1b33"
Sep 09 12:49:32.407035 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:49:32.407014    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"455e5760d531fafabe1ea740b44be53b96b42dcc482908cd50b3918cafcb1b33\": container with ID starting with 455e5760d531fafabe1ea740b44be53b96b42dcc482908cd50b3918cafcb1b33 not found: ID does not exist" containerID="455e5760d531fafabe1ea740b44be53b96b42dcc482908cd50b3918cafcb1b33"
Sep 09 12:49:32.407126 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:32.407042    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"455e5760d531fafabe1ea740b44be53b96b42dcc482908cd50b3918cafcb1b33"} err="failed to get container status \"455e5760d531fafabe1ea740b44be53b96b42dcc482908cd50b3918cafcb1b33\": rpc error: code = NotFound desc = could not find container \"455e5760d531fafabe1ea740b44be53b96b42dcc482908cd50b3918cafcb1b33\": container with ID starting with 455e5760d531fafabe1ea740b44be53b96b42dcc482908cd50b3918cafcb1b33 not found: ID does not exist"
Sep 09 12:49:32.407126 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:32.407058    2582 scope.go:117] "RemoveContainer" containerID="2ac3374065625618a5767884383a6b5572022fdb5d783e050e4858f41d01602b"
Sep 09 12:49:32.407293 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:49:32.407274    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ac3374065625618a5767884383a6b5572022fdb5d783e050e4858f41d01602b\": container with ID starting with 2ac3374065625618a5767884383a6b5572022fdb5d783e050e4858f41d01602b not found: ID does not exist" containerID="2ac3374065625618a5767884383a6b5572022fdb5d783e050e4858f41d01602b"
Sep 09 12:49:32.407340 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:32.407298    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ac3374065625618a5767884383a6b5572022fdb5d783e050e4858f41d01602b"} err="failed to get container status \"2ac3374065625618a5767884383a6b5572022fdb5d783e050e4858f41d01602b\": rpc error: code = NotFound desc = could not find container \"2ac3374065625618a5767884383a6b5572022fdb5d783e050e4858f41d01602b\": container with ID starting with 2ac3374065625618a5767884383a6b5572022fdb5d783e050e4858f41d01602b not found: ID does not exist"
Sep 09 12:49:32.409112 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:32.409093    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"]
Sep 09 12:49:32.412673 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:32.412653    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-610d01-openshift-default-56cf6ffcdb-jwt8b"]
Sep 09 12:49:32.825393 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:32.825364    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7193f5e-78cd-47da-9a51-d579ba099780" path="/var/lib/kubelet/pods/b7193f5e-78cd-47da-9a51-d579ba099780/volumes"
Sep 09 12:49:55.460235 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:55.460201    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"]
Sep 09 12:49:55.460744 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:55.460428    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n" podUID="20dd09ba-9e9f-4e97-a165-9f1a75ce4e83" containerName="maas-api" containerID="cri-o://54551dcbda3067f362ff48b5114ba0d04c08ffeec29888e07b0257fd65dca0bd" gracePeriod=30
Sep 09 12:49:55.726998 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:55.726924    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:49:55.840353 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:55.840325    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-maas-api-tls\") pod \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") "
Sep 09 12:49:55.840514 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:55.840373    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-openshift-service-ca\") pod \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") "
Sep 09 12:49:55.840514 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:55.840420    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5r978\" (UniqueName: \"kubernetes.io/projected/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-kube-api-access-5r978\") pod \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") "
Sep 09 12:49:55.840594 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:55.840512    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-metrics-certs\") pod \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\" (UID: \"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83\") "
Sep 09 12:49:55.840821 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:55.840795    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "20dd09ba-9e9f-4e97-a165-9f1a75ce4e83" (UID: "20dd09ba-9e9f-4e97-a165-9f1a75ce4e83"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:49:55.842731 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:55.842699    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "20dd09ba-9e9f-4e97-a165-9f1a75ce4e83" (UID: "20dd09ba-9e9f-4e97-a165-9f1a75ce4e83"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:49:55.843102 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:55.843077    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-kube-api-access-5r978" (OuterVolumeSpecName: "kube-api-access-5r978") pod "20dd09ba-9e9f-4e97-a165-9f1a75ce4e83" (UID: "20dd09ba-9e9f-4e97-a165-9f1a75ce4e83"). InnerVolumeSpecName "kube-api-access-5r978". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:49:55.843167 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:55.843118    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "20dd09ba-9e9f-4e97-a165-9f1a75ce4e83" (UID: "20dd09ba-9e9f-4e97-a165-9f1a75ce4e83"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:49:55.942110 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:55.942070    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-metrics-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:49:55.942110 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:55.942102    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-maas-api-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:49:55.942353 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:55.942119    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-openshift-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:49:55.942353 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:55.942137    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5r978\" (UniqueName: \"kubernetes.io/projected/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83-kube-api-access-5r978\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:49:56.530688 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:56.530650    2582 generic.go:356] "Generic (PLEG): container finished" podID="20dd09ba-9e9f-4e97-a165-9f1a75ce4e83" containerID="54551dcbda3067f362ff48b5114ba0d04c08ffeec29888e07b0257fd65dca0bd" exitCode=0
Sep 09 12:49:56.531140 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:56.530699    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"
Sep 09 12:49:56.531140 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:56.530733    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n" event={"ID":"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83","Type":"ContainerDied","Data":"54551dcbda3067f362ff48b5114ba0d04c08ffeec29888e07b0257fd65dca0bd"}
Sep 09 12:49:56.531140 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:56.530787    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n" event={"ID":"20dd09ba-9e9f-4e97-a165-9f1a75ce4e83","Type":"ContainerDied","Data":"fc00e7773ad439ab6039ffd2f2e145ae493c30856f6be628c42b1d84c9ec31c1"}
Sep 09 12:49:56.531140 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:56.530812    2582 scope.go:117] "RemoveContainer" containerID="54551dcbda3067f362ff48b5114ba0d04c08ffeec29888e07b0257fd65dca0bd"
Sep 09 12:49:56.540844 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:56.540821    2582 scope.go:117] "RemoveContainer" containerID="54551dcbda3067f362ff48b5114ba0d04c08ffeec29888e07b0257fd65dca0bd"
Sep 09 12:49:56.541123 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:49:56.541093    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54551dcbda3067f362ff48b5114ba0d04c08ffeec29888e07b0257fd65dca0bd\": container with ID starting with 54551dcbda3067f362ff48b5114ba0d04c08ffeec29888e07b0257fd65dca0bd not found: ID does not exist" containerID="54551dcbda3067f362ff48b5114ba0d04c08ffeec29888e07b0257fd65dca0bd"
Sep 09 12:49:56.541179 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:56.541121    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54551dcbda3067f362ff48b5114ba0d04c08ffeec29888e07b0257fd65dca0bd"} err="failed to get container status \"54551dcbda3067f362ff48b5114ba0d04c08ffeec29888e07b0257fd65dca0bd\": rpc error: code = NotFound desc = could not find container \"54551dcbda3067f362ff48b5114ba0d04c08ffeec29888e07b0257fd65dca0bd\": container with ID starting with 54551dcbda3067f362ff48b5114ba0d04c08ffeec29888e07b0257fd65dca0bd not found: ID does not exist"
Sep 09 12:49:56.554253 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:56.554230    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"]
Sep 09 12:49:56.557190 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:56.557169    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-283adb-7987c7f586-dpf9n"]
Sep 09 12:49:56.825285 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:49:56.825200    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20dd09ba-9e9f-4e97-a165-9f1a75ce4e83" path="/var/lib/kubelet/pods/20dd09ba-9e9f-4e97-a165-9f1a75ce4e83/volumes"
Sep 09 12:50:21.026723 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:21.026685    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"]
Sep 09 12:50:21.027218 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:21.026959    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9" podUID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" containerName="istio-proxy" containerID="cri-o://fea7a5193bbc7273f66ec6534feea6e2af0e0ae0e16009c9d38fa2df5e646b6d" gracePeriod=30
Sep 09 12:50:23.641357 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:23.641314    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78bcb7bc6c-5xrn6"]
Sep 09 12:50:23.641862 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:23.641581    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6" podUID="6c8fb8bd-97f3-441b-ac71-6a3efc7054e3" containerName="manager" containerID="cri-o://54db88236d76f3b233c8d83a57600081222cea5df42789fb74e4aee9a05586af" gracePeriod=30
Sep 09 12:50:23.900756 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:23.900688    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6"
Sep 09 12:50:24.001947 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.001892    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpfsw\" (UniqueName: \"kubernetes.io/projected/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-kube-api-access-lpfsw\") pod \"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3\" (UID: \"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3\") "
Sep 09 12:50:24.002116 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.002013    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-metrics-certs\") pod \"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3\" (UID: \"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3\") "
Sep 09 12:50:24.002116 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.002053    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-webhook-cert\") pod \"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3\" (UID: \"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3\") "
Sep 09 12:50:24.004309 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.004282    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-kube-api-access-lpfsw" (OuterVolumeSpecName: "kube-api-access-lpfsw") pod "6c8fb8bd-97f3-441b-ac71-6a3efc7054e3" (UID: "6c8fb8bd-97f3-441b-ac71-6a3efc7054e3"). InnerVolumeSpecName "kube-api-access-lpfsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:50:24.004450 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.004406    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6c8fb8bd-97f3-441b-ac71-6a3efc7054e3" (UID: "6c8fb8bd-97f3-441b-ac71-6a3efc7054e3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:50:24.004612 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.004597    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6c8fb8bd-97f3-441b-ac71-6a3efc7054e3" (UID: "6c8fb8bd-97f3-441b-ac71-6a3efc7054e3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:50:24.104018 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.103970    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpfsw\" (UniqueName: \"kubernetes.io/projected/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-kube-api-access-lpfsw\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:50:24.104018 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.104010    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-metrics-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:50:24.104018 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.104025    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3-webhook-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:50:24.696323 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.696289    2582 generic.go:356] "Generic (PLEG): container finished" podID="6c8fb8bd-97f3-441b-ac71-6a3efc7054e3" containerID="54db88236d76f3b233c8d83a57600081222cea5df42789fb74e4aee9a05586af" exitCode=0
Sep 09 12:50:24.696824 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.696380    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6"
Sep 09 12:50:24.696824 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.696383    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6" event={"ID":"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3","Type":"ContainerDied","Data":"54db88236d76f3b233c8d83a57600081222cea5df42789fb74e4aee9a05586af"}
Sep 09 12:50:24.696824 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.696489    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bcb7bc6c-5xrn6" event={"ID":"6c8fb8bd-97f3-441b-ac71-6a3efc7054e3","Type":"ContainerDied","Data":"4ea1eb1fb7438306f41c06785dcc628d01a1dda5cad414ed2d87afe0f0d8176a"}
Sep 09 12:50:24.696824 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.696524    2582 scope.go:117] "RemoveContainer" containerID="54db88236d76f3b233c8d83a57600081222cea5df42789fb74e4aee9a05586af"
Sep 09 12:50:24.706710 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.706692    2582 scope.go:117] "RemoveContainer" containerID="54db88236d76f3b233c8d83a57600081222cea5df42789fb74e4aee9a05586af"
Sep 09 12:50:24.707031 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:50:24.707007    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54db88236d76f3b233c8d83a57600081222cea5df42789fb74e4aee9a05586af\": container with ID starting with 54db88236d76f3b233c8d83a57600081222cea5df42789fb74e4aee9a05586af not found: ID does not exist" containerID="54db88236d76f3b233c8d83a57600081222cea5df42789fb74e4aee9a05586af"
Sep 09 12:50:24.707112 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.707042    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54db88236d76f3b233c8d83a57600081222cea5df42789fb74e4aee9a05586af"} err="failed to get container status \"54db88236d76f3b233c8d83a57600081222cea5df42789fb74e4aee9a05586af\": rpc error: code = NotFound desc = could not find container \"54db88236d76f3b233c8d83a57600081222cea5df42789fb74e4aee9a05586af\": container with ID starting with 54db88236d76f3b233c8d83a57600081222cea5df42789fb74e4aee9a05586af not found: ID does not exist"
Sep 09 12:50:24.719757 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.719729    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78bcb7bc6c-5xrn6"]
Sep 09 12:50:24.724951 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.724930    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-78bcb7bc6c-5xrn6"]
Sep 09 12:50:24.825581 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:24.825539    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c8fb8bd-97f3-441b-ac71-6a3efc7054e3" path="/var/lib/kubelet/pods/6c8fb8bd-97f3-441b-ac71-6a3efc7054e3/volumes"
Sep 09 12:50:26.292424 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.292349    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:50:26.325268 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.325243    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istiod-ca-cert\") pod \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") "
Sep 09 12:50:26.325420 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.325277    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-token\") pod \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") "
Sep 09 12:50:26.325420 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.325317    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-data\") pod \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") "
Sep 09 12:50:26.325538 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.325499    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lsfb\" (UniqueName: \"kubernetes.io/projected/e7b11f7d-e246-49fb-98eb-5ebf504618dd-kube-api-access-7lsfb\") pod \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") "
Sep 09 12:50:26.325599 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.325541    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-podinfo\") pod \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") "
Sep 09 12:50:26.325655 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.325607    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-credential-socket\") pod \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") "
Sep 09 12:50:26.325655 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.325616    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-data" (OuterVolumeSpecName: "istio-data") pod "e7b11f7d-e246-49fb-98eb-5ebf504618dd" (UID: "e7b11f7d-e246-49fb-98eb-5ebf504618dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:50:26.325764 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.325661    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-workload-socket\") pod \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") "
Sep 09 12:50:26.325764 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.325701    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e7b11f7d-e246-49fb-98eb-5ebf504618dd" (UID: "e7b11f7d-e246-49fb-98eb-5ebf504618dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:50:26.325764 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.325714    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-workload-certs\") pod \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") "
Sep 09 12:50:26.325764 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.325741    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-envoy\") pod \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\" (UID: \"e7b11f7d-e246-49fb-98eb-5ebf504618dd\") "
Sep 09 12:50:26.326071 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.326026    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e7b11f7d-e246-49fb-98eb-5ebf504618dd" (UID: "e7b11f7d-e246-49fb-98eb-5ebf504618dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:50:26.326228 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.326130    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e7b11f7d-e246-49fb-98eb-5ebf504618dd" (UID: "e7b11f7d-e246-49fb-98eb-5ebf504618dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:50:26.326334 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.326165    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e7b11f7d-e246-49fb-98eb-5ebf504618dd" (UID: "e7b11f7d-e246-49fb-98eb-5ebf504618dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:50:26.326500 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.326464    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-credential-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:50:26.326574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.326505    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-workload-socket\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:50:26.326574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.326522    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istiod-ca-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:50:26.326574 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.326536    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:50:26.328185 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.328160    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "e7b11f7d-e246-49fb-98eb-5ebf504618dd" (UID: "e7b11f7d-e246-49fb-98eb-5ebf504618dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:50:26.328185 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.328160    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7b11f7d-e246-49fb-98eb-5ebf504618dd-kube-api-access-7lsfb" (OuterVolumeSpecName: "kube-api-access-7lsfb") pod "e7b11f7d-e246-49fb-98eb-5ebf504618dd" (UID: "e7b11f7d-e246-49fb-98eb-5ebf504618dd"). InnerVolumeSpecName "kube-api-access-7lsfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:50:26.328604 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.328579    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e7b11f7d-e246-49fb-98eb-5ebf504618dd" (UID: "e7b11f7d-e246-49fb-98eb-5ebf504618dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:50:26.328685 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.328606    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e7b11f7d-e246-49fb-98eb-5ebf504618dd" (UID: "e7b11f7d-e246-49fb-98eb-5ebf504618dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:50:26.427188 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.427155    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-workload-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:50:26.427188 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.427183    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-envoy\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:50:26.427352 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.427196    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:50:26.427352 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.427208    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lsfb\" (UniqueName: \"kubernetes.io/projected/e7b11f7d-e246-49fb-98eb-5ebf504618dd-kube-api-access-7lsfb\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:50:26.427352 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.427220    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7b11f7d-e246-49fb-98eb-5ebf504618dd-istio-podinfo\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:50:26.715251 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.715152    2582 generic.go:356] "Generic (PLEG): container finished" podID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" containerID="fea7a5193bbc7273f66ec6534feea6e2af0e0ae0e16009c9d38fa2df5e646b6d" exitCode=0
Sep 09 12:50:26.715538 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.715264    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9" event={"ID":"e7b11f7d-e246-49fb-98eb-5ebf504618dd","Type":"ContainerDied","Data":"fea7a5193bbc7273f66ec6534feea6e2af0e0ae0e16009c9d38fa2df5e646b6d"}
Sep 09 12:50:26.715538 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.715317    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9" event={"ID":"e7b11f7d-e246-49fb-98eb-5ebf504618dd","Type":"ContainerDied","Data":"7366b73eca73d21a3811cb06b0ec4a089bc86c178d937fd110db87ac793c2559"}
Sep 09 12:50:26.715538 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.715368    2582 scope.go:117] "RemoveContainer" containerID="fea7a5193bbc7273f66ec6534feea6e2af0e0ae0e16009c9d38fa2df5e646b6d"
Sep 09 12:50:26.715538 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.715403    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"
Sep 09 12:50:26.726215 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.726190    2582 scope.go:117] "RemoveContainer" containerID="6ad1840aaa28fcb1d37a27834d198b3d30ea603ad4bc661983187315558dba51"
Sep 09 12:50:26.737062 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.737043    2582 scope.go:117] "RemoveContainer" containerID="fea7a5193bbc7273f66ec6534feea6e2af0e0ae0e16009c9d38fa2df5e646b6d"
Sep 09 12:50:26.737315 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:50:26.737296    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fea7a5193bbc7273f66ec6534feea6e2af0e0ae0e16009c9d38fa2df5e646b6d\": container with ID starting with fea7a5193bbc7273f66ec6534feea6e2af0e0ae0e16009c9d38fa2df5e646b6d not found: ID does not exist" containerID="fea7a5193bbc7273f66ec6534feea6e2af0e0ae0e16009c9d38fa2df5e646b6d"
Sep 09 12:50:26.737366 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.737321    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fea7a5193bbc7273f66ec6534feea6e2af0e0ae0e16009c9d38fa2df5e646b6d"} err="failed to get container status \"fea7a5193bbc7273f66ec6534feea6e2af0e0ae0e16009c9d38fa2df5e646b6d\": rpc error: code = NotFound desc = could not find container \"fea7a5193bbc7273f66ec6534feea6e2af0e0ae0e16009c9d38fa2df5e646b6d\": container with ID starting with fea7a5193bbc7273f66ec6534feea6e2af0e0ae0e16009c9d38fa2df5e646b6d not found: ID does not exist"
Sep 09 12:50:26.737366 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.737336    2582 scope.go:117] "RemoveContainer" containerID="6ad1840aaa28fcb1d37a27834d198b3d30ea603ad4bc661983187315558dba51"
Sep 09 12:50:26.737646 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:50:26.737625    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ad1840aaa28fcb1d37a27834d198b3d30ea603ad4bc661983187315558dba51\": container with ID starting with 6ad1840aaa28fcb1d37a27834d198b3d30ea603ad4bc661983187315558dba51 not found: ID does not exist" containerID="6ad1840aaa28fcb1d37a27834d198b3d30ea603ad4bc661983187315558dba51"
Sep 09 12:50:26.737702 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.737652    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ad1840aaa28fcb1d37a27834d198b3d30ea603ad4bc661983187315558dba51"} err="failed to get container status \"6ad1840aaa28fcb1d37a27834d198b3d30ea603ad4bc661983187315558dba51\": rpc error: code = NotFound desc = could not find container \"6ad1840aaa28fcb1d37a27834d198b3d30ea603ad4bc661983187315558dba51\": container with ID starting with 6ad1840aaa28fcb1d37a27834d198b3d30ea603ad4bc661983187315558dba51 not found: ID does not exist"
Sep 09 12:50:26.742063 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.742037    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"]
Sep 09 12:50:26.745666 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.745641    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fe6606-openshift-default-6fc4df845b-6r2q9"]
Sep 09 12:50:26.828407 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:50:26.828370    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" path="/var/lib/kubelet/pods/e7b11f7d-e246-49fb-98eb-5ebf504618dd/volumes"
Sep 09 12:51:25.513532 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.513495    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-78bcb7bc6c-x9mnt"]
Sep 09 12:51:25.515033 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.515005    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7193f5e-78cd-47da-9a51-d579ba099780" containerName="istio-proxy"
Sep 09 12:51:25.515219 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.515205    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7193f5e-78cd-47da-9a51-d579ba099780" containerName="istio-proxy"
Sep 09 12:51:25.515311 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.515300    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" containerName="istio-proxy"
Sep 09 12:51:25.515396 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.515385    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" containerName="istio-proxy"
Sep 09 12:51:25.515476 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.515466    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerName="istio-proxy"
Sep 09 12:51:25.515554 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.515545    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerName="istio-proxy"
Sep 09 12:51:25.515645 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.515635    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c8fb8bd-97f3-441b-ac71-6a3efc7054e3" containerName="manager"
Sep 09 12:51:25.515720 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.515710    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c8fb8bd-97f3-441b-ac71-6a3efc7054e3" containerName="manager"
Sep 09 12:51:25.515795 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.515784    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0af56795-b93a-4d84-b923-d3857d1083e1" containerName="maas-api"
Sep 09 12:51:25.515906 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.515897    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0af56795-b93a-4d84-b923-d3857d1083e1" containerName="maas-api"
Sep 09 12:51:25.515993 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.515983    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerName="istio-proxy"
Sep 09 12:51:25.516065 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.516057    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerName="istio-proxy"
Sep 09 12:51:25.516157 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.516148    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" containerName="istio-proxy"
Sep 09 12:51:25.516228 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.516220    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" containerName="istio-proxy"
Sep 09 12:51:25.516316 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.516307    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b282108d-ba10-48e8-b23e-de118dc5599d" containerName="main"
Sep 09 12:51:25.516388 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.516379    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b282108d-ba10-48e8-b23e-de118dc5599d" containerName="main"
Sep 09 12:51:25.516461 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.516452    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20dd09ba-9e9f-4e97-a165-9f1a75ce4e83" containerName="maas-api"
Sep 09 12:51:25.516533 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.516524    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="20dd09ba-9e9f-4e97-a165-9f1a75ce4e83" containerName="maas-api"
Sep 09 12:51:25.516643 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.516633    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7193f5e-78cd-47da-9a51-d579ba099780" containerName="istio-proxy"
Sep 09 12:51:25.516722 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.516713    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7193f5e-78cd-47da-9a51-d579ba099780" containerName="istio-proxy"
Sep 09 12:51:25.517028 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.517011    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" containerName="istio-proxy"
Sep 09 12:51:25.517138 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.517128    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b282108d-ba10-48e8-b23e-de118dc5599d" containerName="main"
Sep 09 12:51:25.517229 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.517220    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="20dd09ba-9e9f-4e97-a165-9f1a75ce4e83" containerName="maas-api"
Sep 09 12:51:25.517306 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.517297    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerName="istio-proxy"
Sep 09 12:51:25.517445 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.517435    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7193f5e-78cd-47da-9a51-d579ba099780" containerName="istio-proxy"
Sep 09 12:51:25.517529 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.517519    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0af56795-b93a-4d84-b923-d3857d1083e1" containerName="maas-api"
Sep 09 12:51:25.517604 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.517592    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7193f5e-78cd-47da-9a51-d579ba099780" containerName="istio-proxy"
Sep 09 12:51:25.517681 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.517672    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c8fb8bd-97f3-441b-ac71-6a3efc7054e3" containerName="manager"
Sep 09 12:51:25.522109 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.522085    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt"
Sep 09 12:51:25.525004 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.524979    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 12:51:25.525189 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.524979    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m7bgd\""
Sep 09 12:51:25.525189 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.524981    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 12:51:25.525189 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.524991    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 12:51:25.526705 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.526682    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78bcb7bc6c-x9mnt"]
Sep 09 12:51:25.536239 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.536214    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnssj\" (UniqueName: \"kubernetes.io/projected/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-kube-api-access-nnssj\") pod \"maas-controller-78bcb7bc6c-x9mnt\" (UID: \"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5\") " pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt"
Sep 09 12:51:25.536421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.536260    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-metrics-certs\") pod \"maas-controller-78bcb7bc6c-x9mnt\" (UID: \"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5\") " pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt"
Sep 09 12:51:25.536421 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.536398    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-webhook-cert\") pod \"maas-controller-78bcb7bc6c-x9mnt\" (UID: \"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5\") " pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt"
Sep 09 12:51:25.637455 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.637430    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnssj\" (UniqueName: \"kubernetes.io/projected/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-kube-api-access-nnssj\") pod \"maas-controller-78bcb7bc6c-x9mnt\" (UID: \"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5\") " pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt"
Sep 09 12:51:25.637619 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.637463    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-metrics-certs\") pod \"maas-controller-78bcb7bc6c-x9mnt\" (UID: \"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5\") " pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt"
Sep 09 12:51:25.637619 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.637513    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-webhook-cert\") pod \"maas-controller-78bcb7bc6c-x9mnt\" (UID: \"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5\") " pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt"
Sep 09 12:51:25.640036 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.640015    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-metrics-certs\") pod \"maas-controller-78bcb7bc6c-x9mnt\" (UID: \"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5\") " pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt"
Sep 09 12:51:25.640145 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.640054    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-webhook-cert\") pod \"maas-controller-78bcb7bc6c-x9mnt\" (UID: \"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5\") " pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt"
Sep 09 12:51:25.645651 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.645629    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnssj\" (UniqueName: \"kubernetes.io/projected/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-kube-api-access-nnssj\") pod \"maas-controller-78bcb7bc6c-x9mnt\" (UID: \"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5\") " pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt"
Sep 09 12:51:25.834832 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:25.834751    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt"
Sep 09 12:51:26.107271 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:26.107243    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78bcb7bc6c-x9mnt"]
Sep 09 12:51:26.108200 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:51:26.108168    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fc9c1fd_ff9b_40b8_ac89_cc75512ec2c5.slice/crio-628717dca103bdc5d454d6f4d0a8ed4944c9013b7b178041a2937076993d02c8 WatchSource:0}: Error finding container 628717dca103bdc5d454d6f4d0a8ed4944c9013b7b178041a2937076993d02c8: Status 404 returned error can't find the container with id 628717dca103bdc5d454d6f4d0a8ed4944c9013b7b178041a2937076993d02c8
Sep 09 12:51:26.109522 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:26.109503    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:51:27.049061 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:27.049026    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt" event={"ID":"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5","Type":"ContainerStarted","Data":"4832479a88b2bfedd89fc6ca46a46c480080389764bc8ab8c53fe67b252ebe03"}
Sep 09 12:51:27.049412 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:27.049089    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt" event={"ID":"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5","Type":"ContainerStarted","Data":"628717dca103bdc5d454d6f4d0a8ed4944c9013b7b178041a2937076993d02c8"}
Sep 09 12:51:27.049412 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:27.049116    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt"
Sep 09 12:51:27.066078 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:27.066033    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt" podStartSLOduration=1.548370449 podStartE2EDuration="2.066014536s" totalImagesPullingTime="517.644087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:51:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:51:27.064994071 +0000 UTC m=+1906.761559223" watchObservedRunningTime="2026-09-09 12:51:27.066014536 +0000 UTC m=+1906.762579689"
Sep 09 12:51:38.060112 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:38.060083    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt"
Sep 09 12:51:47.396493 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:47.396446    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78bcb7bc6c-x9mnt"]
Sep 09 12:51:47.396985 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:47.396703    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt" podUID="8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5" containerName="manager" containerID="cri-o://4832479a88b2bfedd89fc6ca46a46c480080389764bc8ab8c53fe67b252ebe03" gracePeriod=30
Sep 09 12:51:47.636646 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:47.636621    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt"
Sep 09 12:51:47.713520 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:47.713435    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-metrics-certs\") pod \"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5\" (UID: \"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5\") "
Sep 09 12:51:47.713520 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:47.713487    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-webhook-cert\") pod \"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5\" (UID: \"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5\") "
Sep 09 12:51:47.713729 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:47.713606    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnssj\" (UniqueName: \"kubernetes.io/projected/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-kube-api-access-nnssj\") pod \"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5\" (UID: \"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5\") "
Sep 09 12:51:47.715704 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:47.715667    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5" (UID: "8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:51:47.715840 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:47.715738    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-kube-api-access-nnssj" (OuterVolumeSpecName: "kube-api-access-nnssj") pod "8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5" (UID: "8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5"). InnerVolumeSpecName "kube-api-access-nnssj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:51:47.715840 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:47.715758    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5" (UID: "8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:51:47.814918 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:47.814887    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-metrics-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:51:47.814918 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:47.814911    2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-webhook-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:51:47.815084 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:47.814921    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnssj\" (UniqueName: \"kubernetes.io/projected/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5-kube-api-access-nnssj\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:51:48.180406 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.180371    2582 generic.go:356] "Generic (PLEG): container finished" podID="8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5" containerID="4832479a88b2bfedd89fc6ca46a46c480080389764bc8ab8c53fe67b252ebe03" exitCode=0
Sep 09 12:51:48.180591 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.180432    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt"
Sep 09 12:51:48.180591 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.180464    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt" event={"ID":"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5","Type":"ContainerDied","Data":"4832479a88b2bfedd89fc6ca46a46c480080389764bc8ab8c53fe67b252ebe03"}
Sep 09 12:51:48.180591 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.180540    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bcb7bc6c-x9mnt" event={"ID":"8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5","Type":"ContainerDied","Data":"628717dca103bdc5d454d6f4d0a8ed4944c9013b7b178041a2937076993d02c8"}
Sep 09 12:51:48.180591 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.180576    2582 scope.go:117] "RemoveContainer" containerID="4832479a88b2bfedd89fc6ca46a46c480080389764bc8ab8c53fe67b252ebe03"
Sep 09 12:51:48.191537 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.191459    2582 scope.go:117] "RemoveContainer" containerID="4832479a88b2bfedd89fc6ca46a46c480080389764bc8ab8c53fe67b252ebe03"
Sep 09 12:51:48.192027 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:51:48.192006    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4832479a88b2bfedd89fc6ca46a46c480080389764bc8ab8c53fe67b252ebe03\": container with ID starting with 4832479a88b2bfedd89fc6ca46a46c480080389764bc8ab8c53fe67b252ebe03 not found: ID does not exist" containerID="4832479a88b2bfedd89fc6ca46a46c480080389764bc8ab8c53fe67b252ebe03"
Sep 09 12:51:48.192122 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.192039    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4832479a88b2bfedd89fc6ca46a46c480080389764bc8ab8c53fe67b252ebe03"} err="failed to get container status \"4832479a88b2bfedd89fc6ca46a46c480080389764bc8ab8c53fe67b252ebe03\": rpc error: code = NotFound desc = could not find container \"4832479a88b2bfedd89fc6ca46a46c480080389764bc8ab8c53fe67b252ebe03\": container with ID starting with 4832479a88b2bfedd89fc6ca46a46c480080389764bc8ab8c53fe67b252ebe03 not found: ID does not exist"
Sep 09 12:51:48.207965 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.207925    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78bcb7bc6c-x9mnt"]
Sep 09 12:51:48.211805 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.211779    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-78bcb7bc6c-x9mnt"]
Sep 09 12:51:48.824468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.824435    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5" path="/var/lib/kubelet/pods/8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5/volumes"
Sep 09 12:51:48.985395 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.985366    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-78bcb7bc6c-fhhml"]
Sep 09 12:51:48.985823 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.985806    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5" containerName="manager"
Sep 09 12:51:48.985933 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.985826    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5" containerName="manager"
Sep 09 12:51:48.986011 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.985964    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a47c6ab0-5032-4030-afaa-72a2d9d90c7f" containerName="istio-proxy"
Sep 09 12:51:48.986011 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.985975    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fc9c1fd-ff9b-40b8-ac89-cc75512ec2c5" containerName="manager"
Sep 09 12:51:48.986011 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.985985    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7b11f7d-e246-49fb-98eb-5ebf504618dd" containerName="istio-proxy"
Sep 09 12:51:48.990400 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.990380    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bcb7bc6c-fhhml"
Sep 09 12:51:48.992783 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.992763    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 12:51:48.992907 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.992807    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 12:51:48.992987 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.992912    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 12:51:48.992987 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.992911    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m7bgd\""
Sep 09 12:51:48.996586 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:48.996567    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78bcb7bc6c-fhhml"]
Sep 09 12:51:49.127990 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:49.127910    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp78h\" (UniqueName: \"kubernetes.io/projected/10e3be91-1690-4dc1-ba02-4bdef218d5f7-kube-api-access-tp78h\") pod \"maas-controller-78bcb7bc6c-fhhml\" (UID: \"10e3be91-1690-4dc1-ba02-4bdef218d5f7\") " pod="opendatahub/maas-controller-78bcb7bc6c-fhhml"
Sep 09 12:51:49.128237 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:49.128220    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10e3be91-1690-4dc1-ba02-4bdef218d5f7-metrics-certs\") pod \"maas-controller-78bcb7bc6c-fhhml\" (UID: \"10e3be91-1690-4dc1-ba02-4bdef218d5f7\") " pod="opendatahub/maas-controller-78bcb7bc6c-fhhml"
Sep 09 12:51:49.128390 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:49.128374    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10e3be91-1690-4dc1-ba02-4bdef218d5f7-webhook-cert\") pod \"maas-controller-78bcb7bc6c-fhhml\" (UID: \"10e3be91-1690-4dc1-ba02-4bdef218d5f7\") " pod="opendatahub/maas-controller-78bcb7bc6c-fhhml"
Sep 09 12:51:49.229000 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:49.228977    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp78h\" (UniqueName: \"kubernetes.io/projected/10e3be91-1690-4dc1-ba02-4bdef218d5f7-kube-api-access-tp78h\") pod \"maas-controller-78bcb7bc6c-fhhml\" (UID: \"10e3be91-1690-4dc1-ba02-4bdef218d5f7\") " pod="opendatahub/maas-controller-78bcb7bc6c-fhhml"
Sep 09 12:51:49.229165 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:49.229010    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10e3be91-1690-4dc1-ba02-4bdef218d5f7-metrics-certs\") pod \"maas-controller-78bcb7bc6c-fhhml\" (UID: \"10e3be91-1690-4dc1-ba02-4bdef218d5f7\") " pod="opendatahub/maas-controller-78bcb7bc6c-fhhml"
Sep 09 12:51:49.229165 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:49.229034    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10e3be91-1690-4dc1-ba02-4bdef218d5f7-webhook-cert\") pod \"maas-controller-78bcb7bc6c-fhhml\" (UID: \"10e3be91-1690-4dc1-ba02-4bdef218d5f7\") " pod="opendatahub/maas-controller-78bcb7bc6c-fhhml"
Sep 09 12:51:49.231639 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:49.231618    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10e3be91-1690-4dc1-ba02-4bdef218d5f7-webhook-cert\") pod \"maas-controller-78bcb7bc6c-fhhml\" (UID: \"10e3be91-1690-4dc1-ba02-4bdef218d5f7\") " pod="opendatahub/maas-controller-78bcb7bc6c-fhhml"
Sep 09 12:51:49.231734 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:49.231664    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10e3be91-1690-4dc1-ba02-4bdef218d5f7-metrics-certs\") pod \"maas-controller-78bcb7bc6c-fhhml\" (UID: \"10e3be91-1690-4dc1-ba02-4bdef218d5f7\") " pod="opendatahub/maas-controller-78bcb7bc6c-fhhml"
Sep 09 12:51:49.236325 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:49.236297    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp78h\" (UniqueName: \"kubernetes.io/projected/10e3be91-1690-4dc1-ba02-4bdef218d5f7-kube-api-access-tp78h\") pod \"maas-controller-78bcb7bc6c-fhhml\" (UID: \"10e3be91-1690-4dc1-ba02-4bdef218d5f7\") " pod="opendatahub/maas-controller-78bcb7bc6c-fhhml"
Sep 09 12:51:49.301692 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:49.301658    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bcb7bc6c-fhhml"
Sep 09 12:51:49.573912 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:49.573881    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78bcb7bc6c-fhhml"]
Sep 09 12:51:49.575789 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:51:49.575758    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10e3be91_1690_4dc1_ba02_4bdef218d5f7.slice/crio-52c2e1e12c775e9ad950f40383757779bcd9966884f7ba6edeb4993b2830e8a1 WatchSource:0}: Error finding container 52c2e1e12c775e9ad950f40383757779bcd9966884f7ba6edeb4993b2830e8a1: Status 404 returned error can't find the container with id 52c2e1e12c775e9ad950f40383757779bcd9966884f7ba6edeb4993b2830e8a1
Sep 09 12:51:50.195226 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:50.195196    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bcb7bc6c-fhhml" event={"ID":"10e3be91-1690-4dc1-ba02-4bdef218d5f7","Type":"ContainerStarted","Data":"657118fbcc7da6843ee715c6cfece39baf21d8d65954b593501f94d6da7abdf0"}
Sep 09 12:51:50.195674 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:50.195241    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bcb7bc6c-fhhml" event={"ID":"10e3be91-1690-4dc1-ba02-4bdef218d5f7","Type":"ContainerStarted","Data":"52c2e1e12c775e9ad950f40383757779bcd9966884f7ba6edeb4993b2830e8a1"}
Sep 09 12:51:50.195674 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:50.195337    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-78bcb7bc6c-fhhml"
Sep 09 12:51:50.217868 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:51:50.214972    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-78bcb7bc6c-fhhml" podStartSLOduration=1.801325758 podStartE2EDuration="2.214954043s" totalImagesPullingTime="413.628285ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:51:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:51:50.212239808 +0000 UTC m=+1929.908804981" watchObservedRunningTime="2026-09-09 12:51:50.214954043 +0000 UTC m=+1929.911519196"
Sep 09 12:52:01.202316 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:01.202283    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-78bcb7bc6c-fhhml"
Sep 09 12:52:54.478917 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:54.478884    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"]
Sep 09 12:52:54.479361 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:54.479078    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn" podUID="a6738b5d-a929-438b-bb05-4bfbd39f0c6e" containerName="manager" containerID="cri-o://6eb71b6a83aff647efe5da5ab85f35f23eaa5f9fd1eb4709f98a6429d3e19c2c" gracePeriod=10
Sep 09 12:52:55.938072 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:55.938044    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"
Sep 09 12:52:56.013069 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.013029    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htwp9\" (UniqueName: \"kubernetes.io/projected/a6738b5d-a929-438b-bb05-4bfbd39f0c6e-kube-api-access-htwp9\") pod \"a6738b5d-a929-438b-bb05-4bfbd39f0c6e\" (UID: \"a6738b5d-a929-438b-bb05-4bfbd39f0c6e\") "
Sep 09 12:52:56.013261 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.013089    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6738b5d-a929-438b-bb05-4bfbd39f0c6e-extensions-socket-volume\") pod \"a6738b5d-a929-438b-bb05-4bfbd39f0c6e\" (UID: \"a6738b5d-a929-438b-bb05-4bfbd39f0c6e\") "
Sep 09 12:52:56.013503 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.013478    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6738b5d-a929-438b-bb05-4bfbd39f0c6e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a6738b5d-a929-438b-bb05-4bfbd39f0c6e" (UID: "a6738b5d-a929-438b-bb05-4bfbd39f0c6e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:52:56.015258 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.015232    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6738b5d-a929-438b-bb05-4bfbd39f0c6e-kube-api-access-htwp9" (OuterVolumeSpecName: "kube-api-access-htwp9") pod "a6738b5d-a929-438b-bb05-4bfbd39f0c6e" (UID: "a6738b5d-a929-438b-bb05-4bfbd39f0c6e"). InnerVolumeSpecName "kube-api-access-htwp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:52:56.114822 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.114732    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htwp9\" (UniqueName: \"kubernetes.io/projected/a6738b5d-a929-438b-bb05-4bfbd39f0c6e-kube-api-access-htwp9\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:52:56.114822 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.114764    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6738b5d-a929-438b-bb05-4bfbd39f0c6e-extensions-socket-volume\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:52:56.571391 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.571354    2582 generic.go:356] "Generic (PLEG): container finished" podID="a6738b5d-a929-438b-bb05-4bfbd39f0c6e" containerID="6eb71b6a83aff647efe5da5ab85f35f23eaa5f9fd1eb4709f98a6429d3e19c2c" exitCode=0
Sep 09 12:52:56.571570 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.571403    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"
Sep 09 12:52:56.571570 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.571446    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn" event={"ID":"a6738b5d-a929-438b-bb05-4bfbd39f0c6e","Type":"ContainerDied","Data":"6eb71b6a83aff647efe5da5ab85f35f23eaa5f9fd1eb4709f98a6429d3e19c2c"}
Sep 09 12:52:56.571570 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.571509    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn" event={"ID":"a6738b5d-a929-438b-bb05-4bfbd39f0c6e","Type":"ContainerDied","Data":"c630b1b6b038db3bb5b5fe0cae2828d241bea6862543026ec7f02f983fc4a1eb"}
Sep 09 12:52:56.571570 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.571535    2582 scope.go:117] "RemoveContainer" containerID="6eb71b6a83aff647efe5da5ab85f35f23eaa5f9fd1eb4709f98a6429d3e19c2c"
Sep 09 12:52:56.581748 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.581731    2582 scope.go:117] "RemoveContainer" containerID="6eb71b6a83aff647efe5da5ab85f35f23eaa5f9fd1eb4709f98a6429d3e19c2c"
Sep 09 12:52:56.582067 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:52:56.582046    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eb71b6a83aff647efe5da5ab85f35f23eaa5f9fd1eb4709f98a6429d3e19c2c\": container with ID starting with 6eb71b6a83aff647efe5da5ab85f35f23eaa5f9fd1eb4709f98a6429d3e19c2c not found: ID does not exist" containerID="6eb71b6a83aff647efe5da5ab85f35f23eaa5f9fd1eb4709f98a6429d3e19c2c"
Sep 09 12:52:56.582130 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.582076    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb71b6a83aff647efe5da5ab85f35f23eaa5f9fd1eb4709f98a6429d3e19c2c"} err="failed to get container status \"6eb71b6a83aff647efe5da5ab85f35f23eaa5f9fd1eb4709f98a6429d3e19c2c\": rpc error: code = NotFound desc = could not find container \"6eb71b6a83aff647efe5da5ab85f35f23eaa5f9fd1eb4709f98a6429d3e19c2c\": container with ID starting with 6eb71b6a83aff647efe5da5ab85f35f23eaa5f9fd1eb4709f98a6429d3e19c2c not found: ID does not exist"
Sep 09 12:52:56.604209 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.604181    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"]
Sep 09 12:52:56.615118 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.615092    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wsvqn"]
Sep 09 12:52:56.824875 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:52:56.824765    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6738b5d-a929-438b-bb05-4bfbd39f0c6e" path="/var/lib/kubelet/pods/a6738b5d-a929-438b-bb05-4bfbd39f0c6e/volumes"
Sep 09 12:53:55.223385 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.223349    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz"]
Sep 09 12:53:55.223843 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.223745    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6738b5d-a929-438b-bb05-4bfbd39f0c6e" containerName="manager"
Sep 09 12:53:55.223843 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.223757    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6738b5d-a929-438b-bb05-4bfbd39f0c6e" containerName="manager"
Sep 09 12:53:55.223843 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.223818    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6738b5d-a929-438b-bb05-4bfbd39f0c6e" containerName="manager"
Sep 09 12:53:55.226935 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.226918    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz"
Sep 09 12:53:55.229176 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.229142    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2jbcm\""
Sep 09 12:53:55.237830 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.237810    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz"]
Sep 09 12:53:55.304267 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.304241    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gmv7\" (UniqueName: \"kubernetes.io/projected/a4440c27-0ee7-4dee-98cc-021577dc55c2-kube-api-access-5gmv7\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zcgcz\" (UID: \"a4440c27-0ee7-4dee-98cc-021577dc55c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz"
Sep 09 12:53:55.304369 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.304273    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4440c27-0ee7-4dee-98cc-021577dc55c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zcgcz\" (UID: \"a4440c27-0ee7-4dee-98cc-021577dc55c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz"
Sep 09 12:53:55.405615 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.405591    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gmv7\" (UniqueName: \"kubernetes.io/projected/a4440c27-0ee7-4dee-98cc-021577dc55c2-kube-api-access-5gmv7\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zcgcz\" (UID: \"a4440c27-0ee7-4dee-98cc-021577dc55c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz"
Sep 09 12:53:55.405716 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.405619    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4440c27-0ee7-4dee-98cc-021577dc55c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zcgcz\" (UID: \"a4440c27-0ee7-4dee-98cc-021577dc55c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz"
Sep 09 12:53:55.405977 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.405960    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4440c27-0ee7-4dee-98cc-021577dc55c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zcgcz\" (UID: \"a4440c27-0ee7-4dee-98cc-021577dc55c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz"
Sep 09 12:53:55.416011 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.415988    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gmv7\" (UniqueName: \"kubernetes.io/projected/a4440c27-0ee7-4dee-98cc-021577dc55c2-kube-api-access-5gmv7\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zcgcz\" (UID: \"a4440c27-0ee7-4dee-98cc-021577dc55c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz"
Sep 09 12:53:55.537995 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.537973    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz"
Sep 09 12:53:55.815027 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.815004    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz"]
Sep 09 12:53:55.915723 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.915688    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz" event={"ID":"a4440c27-0ee7-4dee-98cc-021577dc55c2","Type":"ContainerStarted","Data":"819633d76084676e74f87a7883804b53adba87da034fae8a80a39781ebb6beb6"}
Sep 09 12:53:55.915839 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.915745    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz" event={"ID":"a4440c27-0ee7-4dee-98cc-021577dc55c2","Type":"ContainerStarted","Data":"5f7ccdef126581d286f6075b40e169e06a909a8ebc2f8762f8a668fd0d68e34a"}
Sep 09 12:53:55.915839 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.915799    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz"
Sep 09 12:53:55.944245 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:53:55.944200    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz" podStartSLOduration=0.944185553 podStartE2EDuration="944.185553ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:53:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:53:55.941913538 +0000 UTC m=+2055.638478744" watchObservedRunningTime="2026-09-09 12:53:55.944185553 +0000 UTC m=+2055.640750701"
Sep 09 12:54:06.923159 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:06.923080    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zcgcz"
Sep 09 12:54:34.827302 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:34.827269    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"]
Sep 09 12:54:34.830468 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:34.830445    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:34.832316 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:34.832298    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-8e37d975-kserve-self-signed-certs\""
Sep 09 12:54:34.849185 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:34.849166    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"]
Sep 09 12:54:34.920162 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:34.920132    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-dshm\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:34.920274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:34.920180    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k6rr\" (UniqueName: \"kubernetes.io/projected/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-kube-api-access-6k6rr\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:34.920317 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:34.920272    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-tls-certs\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:34.920317 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:34.920304    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-home\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:34.920405 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:34.920322    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-model-cache\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:34.920405 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:34.920353    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-tmp-dir\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:35.021720 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:35.021689    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-tls-certs\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:35.021897 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:35.021727    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-home\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:35.021897 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:35.021752    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-model-cache\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:35.021897 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:35.021783    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-tmp-dir\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:35.021897 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:35.021837    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-dshm\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:35.022156 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:35.021919    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k6rr\" (UniqueName: \"kubernetes.io/projected/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-kube-api-access-6k6rr\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:35.022210 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:35.022165    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-home\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:35.022263 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:35.022204    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-tmp-dir\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:35.022263 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:35.022244    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-model-cache\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:35.024259 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:35.024230    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-dshm\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:35.024442 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:35.024425    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-tls-certs\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:35.030064 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:35.030047    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k6rr\" (UniqueName: \"kubernetes.io/projected/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-kube-api-access-6k6rr\") pod \"e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") " pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:35.140007 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:35.139951    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:35.412736 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:35.412709    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"]
Sep 09 12:54:35.414204 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:54:35.414180    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod908e00ff_51fe_4a7f_b3e6_2c2820e3000a.slice/crio-853071f6de5397b960042118107c5347e99a06662d71ba712320a527fee74f89 WatchSource:0}: Error finding container 853071f6de5397b960042118107c5347e99a06662d71ba712320a527fee74f89: Status 404 returned error can't find the container with id 853071f6de5397b960042118107c5347e99a06662d71ba712320a527fee74f89
Sep 09 12:54:36.135563 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:36.135530    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt" event={"ID":"908e00ff-51fe-4a7f-b3e6-2c2820e3000a","Type":"ContainerStarted","Data":"be753eb523292dc48ed786068a521830c8ec0b2a4b2f3db43db414429040ba58"}
Sep 09 12:54:36.135945 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:36.135573    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt" event={"ID":"908e00ff-51fe-4a7f-b3e6-2c2820e3000a","Type":"ContainerStarted","Data":"853071f6de5397b960042118107c5347e99a06662d71ba712320a527fee74f89"}
Sep 09 12:54:36.152988 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:36.152941    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt" podStartSLOduration=2.152928739 podStartE2EDuration="2.152928739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:54:36.152078997 +0000 UTC m=+2095.848644160" watchObservedRunningTime="2026-09-09 12:54:36.152928739 +0000 UTC m=+2095.849493897"
Sep 09 12:54:45.140289 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:45.140257    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:45.140702 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:45.140303    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:45.152651 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:45.152628    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:54:45.196664 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:54:45.196642    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:55:21.922820 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:21.922779    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"]
Sep 09 12:55:36.944689 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:36.944597    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt" podUID="908e00ff-51fe-4a7f-b3e6-2c2820e3000a" containerName="main" containerID="cri-o://be753eb523292dc48ed786068a521830c8ec0b2a4b2f3db43db414429040ba58" gracePeriod=45
Sep 09 12:55:37.194798 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.194733    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:55:37.272141 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.272103    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k6rr\" (UniqueName: \"kubernetes.io/projected/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-kube-api-access-6k6rr\") pod \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") "
Sep 09 12:55:37.272313 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.272173    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-model-cache\") pod \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") "
Sep 09 12:55:37.272313 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.272212    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-home\") pod \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") "
Sep 09 12:55:37.272313 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.272237    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-dshm\") pod \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") "
Sep 09 12:55:37.272313 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.272278    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-tmp-dir\") pod \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") "
Sep 09 12:55:37.272504 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.272342    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-tls-certs\") pod \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\" (UID: \"908e00ff-51fe-4a7f-b3e6-2c2820e3000a\") "
Sep 09 12:55:37.272559 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.272490    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-model-cache" (OuterVolumeSpecName: "model-cache") pod "908e00ff-51fe-4a7f-b3e6-2c2820e3000a" (UID: "908e00ff-51fe-4a7f-b3e6-2c2820e3000a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:55:37.272559 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.272501    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-home" (OuterVolumeSpecName: "home") pod "908e00ff-51fe-4a7f-b3e6-2c2820e3000a" (UID: "908e00ff-51fe-4a7f-b3e6-2c2820e3000a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:55:37.272660 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.272597    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "908e00ff-51fe-4a7f-b3e6-2c2820e3000a" (UID: "908e00ff-51fe-4a7f-b3e6-2c2820e3000a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:55:37.272695 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.272660    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-model-cache\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:55:37.272695 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.272675    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-home\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:55:37.274499 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.274469    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "908e00ff-51fe-4a7f-b3e6-2c2820e3000a" (UID: "908e00ff-51fe-4a7f-b3e6-2c2820e3000a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:55:37.274631 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.274517    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-kube-api-access-6k6rr" (OuterVolumeSpecName: "kube-api-access-6k6rr") pod "908e00ff-51fe-4a7f-b3e6-2c2820e3000a" (UID: "908e00ff-51fe-4a7f-b3e6-2c2820e3000a"). InnerVolumeSpecName "kube-api-access-6k6rr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:55:37.274631 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.274594    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-dshm" (OuterVolumeSpecName: "dshm") pod "908e00ff-51fe-4a7f-b3e6-2c2820e3000a" (UID: "908e00ff-51fe-4a7f-b3e6-2c2820e3000a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:55:37.374003 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.373966    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-dshm\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:55:37.374003 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.373998    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-tmp-dir\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:55:37.374003 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.374012    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-tls-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:55:37.374254 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.374020    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k6rr\" (UniqueName: \"kubernetes.io/projected/908e00ff-51fe-4a7f-b3e6-2c2820e3000a-kube-api-access-6k6rr\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:55:37.478002 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.477910    2582 generic.go:356] "Generic (PLEG): container finished" podID="908e00ff-51fe-4a7f-b3e6-2c2820e3000a" containerID="be753eb523292dc48ed786068a521830c8ec0b2a4b2f3db43db414429040ba58" exitCode=0
Sep 09 12:55:37.478002 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.477989    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"
Sep 09 12:55:37.478181 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.477995    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt" event={"ID":"908e00ff-51fe-4a7f-b3e6-2c2820e3000a","Type":"ContainerDied","Data":"be753eb523292dc48ed786068a521830c8ec0b2a4b2f3db43db414429040ba58"}
Sep 09 12:55:37.478181 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.478093    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt" event={"ID":"908e00ff-51fe-4a7f-b3e6-2c2820e3000a","Type":"ContainerDied","Data":"853071f6de5397b960042118107c5347e99a06662d71ba712320a527fee74f89"}
Sep 09 12:55:37.478181 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.478110    2582 scope.go:117] "RemoveContainer" containerID="be753eb523292dc48ed786068a521830c8ec0b2a4b2f3db43db414429040ba58"
Sep 09 12:55:37.487635 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.487616    2582 scope.go:117] "RemoveContainer" containerID="be753eb523292dc48ed786068a521830c8ec0b2a4b2f3db43db414429040ba58"
Sep 09 12:55:37.487992 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:55:37.487971    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be753eb523292dc48ed786068a521830c8ec0b2a4b2f3db43db414429040ba58\": container with ID starting with be753eb523292dc48ed786068a521830c8ec0b2a4b2f3db43db414429040ba58 not found: ID does not exist" containerID="be753eb523292dc48ed786068a521830c8ec0b2a4b2f3db43db414429040ba58"
Sep 09 12:55:37.488053 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.488002    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be753eb523292dc48ed786068a521830c8ec0b2a4b2f3db43db414429040ba58"} err="failed to get container status \"be753eb523292dc48ed786068a521830c8ec0b2a4b2f3db43db414429040ba58\": rpc error: code = NotFound desc = could not find container \"be753eb523292dc48ed786068a521830c8ec0b2a4b2f3db43db414429040ba58\": container with ID starting with be753eb523292dc48ed786068a521830c8ec0b2a4b2f3db43db414429040ba58 not found: ID does not exist"
Sep 09 12:55:37.498943 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.498920    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"]
Sep 09 12:55:37.502715 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:37.502691    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-8e37d975-kserve-567c69677f-sfhkt"]
Sep 09 12:55:38.831582 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:38.831535    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="908e00ff-51fe-4a7f-b3e6-2c2820e3000a" path="/var/lib/kubelet/pods/908e00ff-51fe-4a7f-b3e6-2c2820e3000a/volumes"
Sep 09 12:55:42.161550 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.161513    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"]
Sep 09 12:55:42.162150 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.162124    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="908e00ff-51fe-4a7f-b3e6-2c2820e3000a" containerName="main"
Sep 09 12:55:42.162267 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.162170    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="908e00ff-51fe-4a7f-b3e6-2c2820e3000a" containerName="main"
Sep 09 12:55:42.162364 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.162350    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="908e00ff-51fe-4a7f-b3e6-2c2820e3000a" containerName="main"
Sep 09 12:55:42.166820 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.166798    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:42.174934 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.174844    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"]
Sep 09 12:55:42.318258 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.318211    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/818ce25c-99a7-4e49-a249-5faf15adad0e-openshift-service-ca\") pod \"maas-api-8df5f485c-7hpk6\" (UID: \"818ce25c-99a7-4e49-a249-5faf15adad0e\") " pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:42.318440 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.318293    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/818ce25c-99a7-4e49-a249-5faf15adad0e-maas-api-tls\") pod \"maas-api-8df5f485c-7hpk6\" (UID: \"818ce25c-99a7-4e49-a249-5faf15adad0e\") " pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:42.318440 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.318352    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjwrz\" (UniqueName: \"kubernetes.io/projected/818ce25c-99a7-4e49-a249-5faf15adad0e-kube-api-access-xjwrz\") pod \"maas-api-8df5f485c-7hpk6\" (UID: \"818ce25c-99a7-4e49-a249-5faf15adad0e\") " pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:42.318539 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.318446    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/818ce25c-99a7-4e49-a249-5faf15adad0e-metrics-certs\") pod \"maas-api-8df5f485c-7hpk6\" (UID: \"818ce25c-99a7-4e49-a249-5faf15adad0e\") " pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:42.419797 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.419699    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjwrz\" (UniqueName: \"kubernetes.io/projected/818ce25c-99a7-4e49-a249-5faf15adad0e-kube-api-access-xjwrz\") pod \"maas-api-8df5f485c-7hpk6\" (UID: \"818ce25c-99a7-4e49-a249-5faf15adad0e\") " pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:42.419991 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.419796    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/818ce25c-99a7-4e49-a249-5faf15adad0e-metrics-certs\") pod \"maas-api-8df5f485c-7hpk6\" (UID: \"818ce25c-99a7-4e49-a249-5faf15adad0e\") " pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:42.419991 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.419935    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/818ce25c-99a7-4e49-a249-5faf15adad0e-openshift-service-ca\") pod \"maas-api-8df5f485c-7hpk6\" (UID: \"818ce25c-99a7-4e49-a249-5faf15adad0e\") " pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:42.419991 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.419973    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/818ce25c-99a7-4e49-a249-5faf15adad0e-maas-api-tls\") pod \"maas-api-8df5f485c-7hpk6\" (UID: \"818ce25c-99a7-4e49-a249-5faf15adad0e\") " pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:42.420911 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.420809    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/818ce25c-99a7-4e49-a249-5faf15adad0e-openshift-service-ca\") pod \"maas-api-8df5f485c-7hpk6\" (UID: \"818ce25c-99a7-4e49-a249-5faf15adad0e\") " pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:42.422870 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.422827    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/818ce25c-99a7-4e49-a249-5faf15adad0e-maas-api-tls\") pod \"maas-api-8df5f485c-7hpk6\" (UID: \"818ce25c-99a7-4e49-a249-5faf15adad0e\") " pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:42.423192 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.423170    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/818ce25c-99a7-4e49-a249-5faf15adad0e-metrics-certs\") pod \"maas-api-8df5f485c-7hpk6\" (UID: \"818ce25c-99a7-4e49-a249-5faf15adad0e\") " pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:42.428271 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.428244    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjwrz\" (UniqueName: \"kubernetes.io/projected/818ce25c-99a7-4e49-a249-5faf15adad0e-kube-api-access-xjwrz\") pod \"maas-api-8df5f485c-7hpk6\" (UID: \"818ce25c-99a7-4e49-a249-5faf15adad0e\") " pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:42.480135 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.479889    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:42.777166 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:55:42.777132    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod818ce25c_99a7_4e49_a249_5faf15adad0e.slice/crio-e8719a9ec836f1ab508436eb91fdd2b41f7a3db90aa1d0c07e5ee15017e6e56d WatchSource:0}: Error finding container e8719a9ec836f1ab508436eb91fdd2b41f7a3db90aa1d0c07e5ee15017e6e56d: Status 404 returned error can't find the container with id e8719a9ec836f1ab508436eb91fdd2b41f7a3db90aa1d0c07e5ee15017e6e56d
Sep 09 12:55:42.777166 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:42.777153    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"]
Sep 09 12:55:43.523779 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:43.523743    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6" event={"ID":"818ce25c-99a7-4e49-a249-5faf15adad0e","Type":"ContainerStarted","Data":"7a00137a2f3f00782cee4938b7e8d5c72eec4f68796bd0e0fb27197ac8a16435"}
Sep 09 12:55:43.524233 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:43.523909    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:43.524233 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:43.523959    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6" event={"ID":"818ce25c-99a7-4e49-a249-5faf15adad0e","Type":"ContainerStarted","Data":"e8719a9ec836f1ab508436eb91fdd2b41f7a3db90aa1d0c07e5ee15017e6e56d"}
Sep 09 12:55:43.541956 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:43.541898    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6" podStartSLOduration=1.107063652 podStartE2EDuration="1.541884505s" totalImagesPullingTime="434.820853ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:55:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:55:43.540255829 +0000 UTC m=+2163.236820981" watchObservedRunningTime="2026-09-09 12:55:43.541884505 +0000 UTC m=+2163.238449658"
Sep 09 12:55:49.535042 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:49.535006    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8df5f485c-7hpk6"
Sep 09 12:55:49.591422 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:49.591384    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"]
Sep 09 12:55:49.591722 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:49.591665    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2" podUID="1451c9d8-ca35-4a61-b8a9-0912b4692982" containerName="maas-api" containerID="cri-o://7cf8e27e06e2571a98332deb2a01f10b47ffc49809f0df5649d92ae9a318b6f9" gracePeriod=30
Sep 09 12:55:50.864418 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:50.864393    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:55:50.996830 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:50.996800    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1451c9d8-ca35-4a61-b8a9-0912b4692982-openshift-service-ca\") pod \"1451c9d8-ca35-4a61-b8a9-0912b4692982\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") "
Sep 09 12:55:50.997029 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:50.996842    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1451c9d8-ca35-4a61-b8a9-0912b4692982-maas-api-tls\") pod \"1451c9d8-ca35-4a61-b8a9-0912b4692982\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") "
Sep 09 12:55:50.997029 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:50.996937    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lztgt\" (UniqueName: \"kubernetes.io/projected/1451c9d8-ca35-4a61-b8a9-0912b4692982-kube-api-access-lztgt\") pod \"1451c9d8-ca35-4a61-b8a9-0912b4692982\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") "
Sep 09 12:55:50.997029 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:50.996977    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1451c9d8-ca35-4a61-b8a9-0912b4692982-metrics-certs\") pod \"1451c9d8-ca35-4a61-b8a9-0912b4692982\" (UID: \"1451c9d8-ca35-4a61-b8a9-0912b4692982\") "
Sep 09 12:55:50.997203 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:50.997169    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1451c9d8-ca35-4a61-b8a9-0912b4692982-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1451c9d8-ca35-4a61-b8a9-0912b4692982" (UID: "1451c9d8-ca35-4a61-b8a9-0912b4692982"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:55:50.997455 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:50.997427    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1451c9d8-ca35-4a61-b8a9-0912b4692982-openshift-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:55:50.999617 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:50.999589    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1451c9d8-ca35-4a61-b8a9-0912b4692982-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1451c9d8-ca35-4a61-b8a9-0912b4692982" (UID: "1451c9d8-ca35-4a61-b8a9-0912b4692982"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:55:50.999762 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:50.999731    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1451c9d8-ca35-4a61-b8a9-0912b4692982-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1451c9d8-ca35-4a61-b8a9-0912b4692982" (UID: "1451c9d8-ca35-4a61-b8a9-0912b4692982"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:55:50.999910 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:50.999813    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1451c9d8-ca35-4a61-b8a9-0912b4692982-kube-api-access-lztgt" (OuterVolumeSpecName: "kube-api-access-lztgt") pod "1451c9d8-ca35-4a61-b8a9-0912b4692982" (UID: "1451c9d8-ca35-4a61-b8a9-0912b4692982"). InnerVolumeSpecName "kube-api-access-lztgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:55:51.098395 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:51.098364    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lztgt\" (UniqueName: \"kubernetes.io/projected/1451c9d8-ca35-4a61-b8a9-0912b4692982-kube-api-access-lztgt\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:55:51.098395 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:51.098391    2582 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1451c9d8-ca35-4a61-b8a9-0912b4692982-metrics-certs\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:55:51.098395 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:51.098400    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1451c9d8-ca35-4a61-b8a9-0912b4692982-maas-api-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
Sep 09 12:55:51.571619 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:51.571580    2582 generic.go:356] "Generic (PLEG): container finished" podID="1451c9d8-ca35-4a61-b8a9-0912b4692982" containerID="7cf8e27e06e2571a98332deb2a01f10b47ffc49809f0df5649d92ae9a318b6f9" exitCode=0
Sep 09 12:55:51.571619 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:51.571615    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"
Sep 09 12:55:51.571906 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:51.571660    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2" event={"ID":"1451c9d8-ca35-4a61-b8a9-0912b4692982","Type":"ContainerDied","Data":"7cf8e27e06e2571a98332deb2a01f10b47ffc49809f0df5649d92ae9a318b6f9"}
Sep 09 12:55:51.571906 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:51.571724    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2" event={"ID":"1451c9d8-ca35-4a61-b8a9-0912b4692982","Type":"ContainerDied","Data":"972b1dcfd5133641e1c9cd9a238c7343826f391d8edabb0a351f44e09468e91f"}
Sep 09 12:55:51.571906 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:51.571755    2582 scope.go:117] "RemoveContainer" containerID="7cf8e27e06e2571a98332deb2a01f10b47ffc49809f0df5649d92ae9a318b6f9"
Sep 09 12:55:51.581782 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:51.581766    2582 scope.go:117] "RemoveContainer" containerID="7cf8e27e06e2571a98332deb2a01f10b47ffc49809f0df5649d92ae9a318b6f9"
Sep 09 12:55:51.582121 ip-10-0-139-173 kubenswrapper[2582]: E0909 12:55:51.582101    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cf8e27e06e2571a98332deb2a01f10b47ffc49809f0df5649d92ae9a318b6f9\": container with ID starting with 7cf8e27e06e2571a98332deb2a01f10b47ffc49809f0df5649d92ae9a318b6f9 not found: ID does not exist" containerID="7cf8e27e06e2571a98332deb2a01f10b47ffc49809f0df5649d92ae9a318b6f9"
Sep 09 12:55:51.582193 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:51.582130    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cf8e27e06e2571a98332deb2a01f10b47ffc49809f0df5649d92ae9a318b6f9"} err="failed to get container status \"7cf8e27e06e2571a98332deb2a01f10b47ffc49809f0df5649d92ae9a318b6f9\": rpc error: code = NotFound desc = could not find container \"7cf8e27e06e2571a98332deb2a01f10b47ffc49809f0df5649d92ae9a318b6f9\": container with ID starting with 7cf8e27e06e2571a98332deb2a01f10b47ffc49809f0df5649d92ae9a318b6f9 not found: ID does not exist"
Sep 09 12:55:51.595025 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:51.594999    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"]
Sep 09 12:55:51.600994 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:51.600969    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-59675b95bf-qg6b2"]
Sep 09 12:55:52.825143 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:55:52.825106    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1451c9d8-ca35-4a61-b8a9-0912b4692982" path="/var/lib/kubelet/pods/1451c9d8-ca35-4a61-b8a9-0912b4692982/volumes"
Sep 09 12:56:13.195901 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:13.195869    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65c77c4788-b9pxf_f42aff64-617a-4093-b460-a1be1194394a/authorino/0.log"
Sep 09 12:56:18.678057 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:18.678021    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xk9r2_7d39ec25-d666-4150-b545-d3d6bf8c9417/manager/0.log"
Sep 09 12:56:18.799034 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:18.799010    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jn9jk_37efbe2e-4604-48f8-982f-babd48f9aba3/manager/0.log"
Sep 09 12:56:18.929747 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:18.929679    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-78bcb7bc6c-fhhml_10e3be91-1690-4dc1-ba02-4bdef218d5f7/manager/0.log"
Sep 09 12:56:19.558274 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:19.558245    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c5d49f968-x466m_9b86bae9-72d0-48c2-b3c2-91651d80a6d3/manager/0.log"
Sep 09 12:56:19.568228 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:19.568203    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c5d49f968-x466m_9b86bae9-72d0-48c2-b3c2-91651d80a6d3/copy-manifests/0.log"
Sep 09 12:56:19.935710 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:19.935635    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8df5f485c-7hpk6_818ce25c-99a7-4e49-a249-5faf15adad0e/maas-api/0.log"
Sep 09 12:56:20.286988 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:20.286964    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-l77jd_d761a477-dc03-4e3c-8093-26e1053df8f7/postgres/0.log"
Sep 09 12:56:21.101166 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:21.101136    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65c77c4788-b9pxf_f42aff64-617a-4093-b460-a1be1194394a/authorino/0.log"
Sep 09 12:56:21.223469 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:21.223443    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-g2s7l_b96aaf4b-8d2d-4a22-b0f5-6e902ed5b564/manager/0.log"
Sep 09 12:56:21.337839 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:21.337806    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-8lgh8_b173158e-31e1-4351-8cb6-4697d2260a17/manager/0.log"
Sep 09 12:56:21.578867 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:21.578829    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-zcgcz_a4440c27-0ee7-4dee-98cc-021577dc55c2/manager/0.log"
Sep 09 12:56:21.821471 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:21.821443    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-gjpwp_ac115547-fe26-4e18-92ef-1cb8093ac78a/manager/0.log"
Sep 09 12:56:22.541812 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:22.541770    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5948bbb55b-hznrz_6946ad55-ed72-48f5-a38d-a722eac0e7f0/kube-auth-proxy/0.log"
Sep 09 12:56:22.658009 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:22.657984    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-msm2r_4efdd84f-7b2d-43a6-b028-5426f2b68203/istio-proxy/1.log"
Sep 09 12:56:22.906237 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:22.906175    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hclrp_fec7ad7e-69c0-4f61-9c45-4214164224b5/payload-processing/0.log"
Sep 09 12:56:23.617074 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:23.617042    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-njrps_29ebf290-5def-4871-a5e7-6a17e4d3f836/main/0.log"
Sep 09 12:56:29.495701 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.495665    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-prpjx/must-gather-vcjlk"]
Sep 09 12:56:29.496388 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.496364    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1451c9d8-ca35-4a61-b8a9-0912b4692982" containerName="maas-api"
Sep 09 12:56:29.496486 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.496390    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1451c9d8-ca35-4a61-b8a9-0912b4692982" containerName="maas-api"
Sep 09 12:56:29.496548 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.496492    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1451c9d8-ca35-4a61-b8a9-0912b4692982" containerName="maas-api"
Sep 09 12:56:29.499999 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.499976    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prpjx/must-gather-vcjlk"
Sep 09 12:56:29.502121 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.502103    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-prpjx\"/\"kube-root-ca.crt\""
Sep 09 12:56:29.502595 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.502576    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-prpjx\"/\"default-dockercfg-lz68r\""
Sep 09 12:56:29.502647 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.502588    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-prpjx\"/\"openshift-service-ca.crt\""
Sep 09 12:56:29.508121 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.508095    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prpjx/must-gather-vcjlk"]
Sep 09 12:56:29.625193 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.625163    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xngx\" (UniqueName: \"kubernetes.io/projected/0c4c31b5-2b8d-48fb-aa28-cfdca6511fbc-kube-api-access-2xngx\") pod \"must-gather-vcjlk\" (UID: \"0c4c31b5-2b8d-48fb-aa28-cfdca6511fbc\") " pod="openshift-must-gather-prpjx/must-gather-vcjlk"
Sep 09 12:56:29.625318 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.625204    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0c4c31b5-2b8d-48fb-aa28-cfdca6511fbc-must-gather-output\") pod \"must-gather-vcjlk\" (UID: \"0c4c31b5-2b8d-48fb-aa28-cfdca6511fbc\") " pod="openshift-must-gather-prpjx/must-gather-vcjlk"
Sep 09 12:56:29.726165 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.726137    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xngx\" (UniqueName: \"kubernetes.io/projected/0c4c31b5-2b8d-48fb-aa28-cfdca6511fbc-kube-api-access-2xngx\") pod \"must-gather-vcjlk\" (UID: \"0c4c31b5-2b8d-48fb-aa28-cfdca6511fbc\") " pod="openshift-must-gather-prpjx/must-gather-vcjlk"
Sep 09 12:56:29.726284 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.726180    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0c4c31b5-2b8d-48fb-aa28-cfdca6511fbc-must-gather-output\") pod \"must-gather-vcjlk\" (UID: \"0c4c31b5-2b8d-48fb-aa28-cfdca6511fbc\") " pod="openshift-must-gather-prpjx/must-gather-vcjlk"
Sep 09 12:56:29.726556 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.726537    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0c4c31b5-2b8d-48fb-aa28-cfdca6511fbc-must-gather-output\") pod \"must-gather-vcjlk\" (UID: \"0c4c31b5-2b8d-48fb-aa28-cfdca6511fbc\") " pod="openshift-must-gather-prpjx/must-gather-vcjlk"
Sep 09 12:56:29.736177 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.736157    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xngx\" (UniqueName: \"kubernetes.io/projected/0c4c31b5-2b8d-48fb-aa28-cfdca6511fbc-kube-api-access-2xngx\") pod \"must-gather-vcjlk\" (UID: \"0c4c31b5-2b8d-48fb-aa28-cfdca6511fbc\") " pod="openshift-must-gather-prpjx/must-gather-vcjlk"
Sep 09 12:56:29.810749 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:29.810698    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prpjx/must-gather-vcjlk"
Sep 09 12:56:30.083058 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:30.083025    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prpjx/must-gather-vcjlk"]
Sep 09 12:56:30.084566 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:56:30.084540    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c4c31b5_2b8d_48fb_aa28_cfdca6511fbc.slice/crio-529caa53e0c48323e5d5edf0feabb1b8382df489a4f16de1b20816f45c23bc86 WatchSource:0}: Error finding container 529caa53e0c48323e5d5edf0feabb1b8382df489a4f16de1b20816f45c23bc86: Status 404 returned error can't find the container with id 529caa53e0c48323e5d5edf0feabb1b8382df489a4f16de1b20816f45c23bc86
Sep 09 12:56:30.086603 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:30.086587    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:56:30.802158 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:30.802125    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prpjx/must-gather-vcjlk" event={"ID":"0c4c31b5-2b8d-48fb-aa28-cfdca6511fbc","Type":"ContainerStarted","Data":"529caa53e0c48323e5d5edf0feabb1b8382df489a4f16de1b20816f45c23bc86"}
Sep 09 12:56:31.814019 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:31.813980    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prpjx/must-gather-vcjlk" event={"ID":"0c4c31b5-2b8d-48fb-aa28-cfdca6511fbc","Type":"ContainerStarted","Data":"bc084d1bb4aec9a32de3a90ddaf4f33e23a834996d4fba97982ff843b96c82e3"}
Sep 09 12:56:31.814443 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:31.814134    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prpjx/must-gather-vcjlk" event={"ID":"0c4c31b5-2b8d-48fb-aa28-cfdca6511fbc","Type":"ContainerStarted","Data":"8189d325789990f30bc94156a4f2d5e01192de40bf03cbf0801c36c7f7d1c509"}
Sep 09 12:56:31.829826 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:31.829777    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-prpjx/must-gather-vcjlk" podStartSLOduration=1.6758729909999999 podStartE2EDuration="2.829762622s" totalImagesPullingTime="1.153889631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:56:31.828228759 +0000 UTC m=+2211.524793910" watchObservedRunningTime="2026-09-09 12:56:31.829762622 +0000 UTC m=+2211.526327773"
Sep 09 12:56:32.910478 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:32.910446    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lxbw5_cb9acdab-04f6-47b1-9bcd-d24246dcf52a/global-pull-secret-syncer/0.log"
Sep 09 12:56:33.066384 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:33.066356    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tqtbz_5aabdb32-70b9-4bbf-a9ca-b15d76924dc7/konnectivity-agent/0.log"
Sep 09 12:56:33.112434 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:33.112406    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-173.ec2.internal_0512c5589b44ea46a23646a68804fa89/haproxy/0.log"
Sep 09 12:56:37.666874 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:37.666140    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65c77c4788-b9pxf_f42aff64-617a-4093-b460-a1be1194394a/authorino/0.log"
Sep 09 12:56:37.696920 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:37.695880    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-g2s7l_b96aaf4b-8d2d-4a22-b0f5-6e902ed5b564/manager/0.log"
Sep 09 12:56:37.723543 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:37.721216    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-8lgh8_b173158e-31e1-4351-8cb6-4697d2260a17/manager/0.log"
Sep 09 12:56:37.793667 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:37.793636    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-zcgcz_a4440c27-0ee7-4dee-98cc-021577dc55c2/manager/0.log"
Sep 09 12:56:37.899273 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:37.897681    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-gjpwp_ac115547-fe26-4e18-92ef-1cb8093ac78a/manager/0.log"
Sep 09 12:56:39.726152 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:39.726115    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-rqwnc_36f7cc6e-6e5b-472f-b3f9-e1978d880277/kube-state-metrics/0.log"
Sep 09 12:56:39.746494 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:39.746467    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-rqwnc_36f7cc6e-6e5b-472f-b3f9-e1978d880277/kube-rbac-proxy-main/0.log"
Sep 09 12:56:39.771725 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:39.771696    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-rqwnc_36f7cc6e-6e5b-472f-b3f9-e1978d880277/kube-rbac-proxy-self/0.log"
Sep 09 12:56:39.847873 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:39.847801    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-r5ggx_704f3e3e-97a3-41b4-bd6d-c148e147ad40/monitoring-plugin/0.log"
Sep 09 12:56:39.887047 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:39.887018    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ggzw9_c89c9721-240b-4370-90fd-bf1484cef7a6/node-exporter/0.log"
Sep 09 12:56:39.918297 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:39.918256    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ggzw9_c89c9721-240b-4370-90fd-bf1484cef7a6/kube-rbac-proxy/0.log"
Sep 09 12:56:39.950678 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:39.950654    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ggzw9_c89c9721-240b-4370-90fd-bf1484cef7a6/init-textfile/0.log"
Sep 09 12:56:40.194157 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:40.194128    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-w87hx_92925e47-7972-411e-b1c4-7265ff222b6a/kube-rbac-proxy-main/0.log"
Sep 09 12:56:40.217703 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:40.217672    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-w87hx_92925e47-7972-411e-b1c4-7265ff222b6a/kube-rbac-proxy-self/0.log"
Sep 09 12:56:40.240727 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:40.240695    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-w87hx_92925e47-7972-411e-b1c4-7265ff222b6a/openshift-state-metrics/0.log"
Sep 09 12:56:40.461544 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:40.461469    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-pgmhg_47fae105-1e04-4d37-90ed-da1fa03d55b6/prometheus-operator/0.log"
Sep 09 12:56:40.483711 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:40.483679    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-pgmhg_47fae105-1e04-4d37-90ed-da1fa03d55b6/kube-rbac-proxy/0.log"
Sep 09 12:56:40.542455 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:40.542385    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7799b8d7d9-kzb2w_45460890-41ca-41f5-b1a8-4aa4207d23d9/telemeter-client/0.log"
Sep 09 12:56:40.564377 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:40.564318    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7799b8d7d9-kzb2w_45460890-41ca-41f5-b1a8-4aa4207d23d9/reload/0.log"
Sep 09 12:56:40.610481 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:40.610448    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7799b8d7d9-kzb2w_45460890-41ca-41f5-b1a8-4aa4207d23d9/kube-rbac-proxy/0.log"
Sep 09 12:56:42.861097 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:42.861062    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"]
Sep 09 12:56:42.869585 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:42.869555    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:42.872509 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:42.872484    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"]
Sep 09 12:56:42.977559 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:42.977509    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/14829109-4102-4cc0-92cb-95c42f6b2373-lib-modules\") pod \"perf-node-gather-daemonset-msf5w\" (UID: \"14829109-4102-4cc0-92cb-95c42f6b2373\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:42.977757 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:42.977649    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/14829109-4102-4cc0-92cb-95c42f6b2373-proc\") pod \"perf-node-gather-daemonset-msf5w\" (UID: \"14829109-4102-4cc0-92cb-95c42f6b2373\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:42.977833 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:42.977755    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14829109-4102-4cc0-92cb-95c42f6b2373-sys\") pod \"perf-node-gather-daemonset-msf5w\" (UID: \"14829109-4102-4cc0-92cb-95c42f6b2373\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:42.977937 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:42.977845    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/14829109-4102-4cc0-92cb-95c42f6b2373-podres\") pod \"perf-node-gather-daemonset-msf5w\" (UID: \"14829109-4102-4cc0-92cb-95c42f6b2373\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:42.978062 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:42.978025    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6xlm\" (UniqueName: \"kubernetes.io/projected/14829109-4102-4cc0-92cb-95c42f6b2373-kube-api-access-j6xlm\") pod \"perf-node-gather-daemonset-msf5w\" (UID: \"14829109-4102-4cc0-92cb-95c42f6b2373\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:43.079068 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.079029    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/14829109-4102-4cc0-92cb-95c42f6b2373-lib-modules\") pod \"perf-node-gather-daemonset-msf5w\" (UID: \"14829109-4102-4cc0-92cb-95c42f6b2373\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:43.079505 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.079485    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/14829109-4102-4cc0-92cb-95c42f6b2373-proc\") pod \"perf-node-gather-daemonset-msf5w\" (UID: \"14829109-4102-4cc0-92cb-95c42f6b2373\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:43.079766 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.079750    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14829109-4102-4cc0-92cb-95c42f6b2373-sys\") pod \"perf-node-gather-daemonset-msf5w\" (UID: \"14829109-4102-4cc0-92cb-95c42f6b2373\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:43.079990 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.079975    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/14829109-4102-4cc0-92cb-95c42f6b2373-podres\") pod \"perf-node-gather-daemonset-msf5w\" (UID: \"14829109-4102-4cc0-92cb-95c42f6b2373\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:43.080336 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.079415    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/14829109-4102-4cc0-92cb-95c42f6b2373-lib-modules\") pod \"perf-node-gather-daemonset-msf5w\" (UID: \"14829109-4102-4cc0-92cb-95c42f6b2373\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:43.080689 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.079911    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14829109-4102-4cc0-92cb-95c42f6b2373-sys\") pod \"perf-node-gather-daemonset-msf5w\" (UID: \"14829109-4102-4cc0-92cb-95c42f6b2373\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:43.080803 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.080145    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/14829109-4102-4cc0-92cb-95c42f6b2373-podres\") pod \"perf-node-gather-daemonset-msf5w\" (UID: \"14829109-4102-4cc0-92cb-95c42f6b2373\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:43.080922 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.079654    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/14829109-4102-4cc0-92cb-95c42f6b2373-proc\") pod \"perf-node-gather-daemonset-msf5w\" (UID: \"14829109-4102-4cc0-92cb-95c42f6b2373\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:43.081014 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.080429    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6xlm\" (UniqueName: \"kubernetes.io/projected/14829109-4102-4cc0-92cb-95c42f6b2373-kube-api-access-j6xlm\") pod \"perf-node-gather-daemonset-msf5w\" (UID: \"14829109-4102-4cc0-92cb-95c42f6b2373\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:43.088157 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.088138    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6xlm\" (UniqueName: \"kubernetes.io/projected/14829109-4102-4cc0-92cb-95c42f6b2373-kube-api-access-j6xlm\") pod \"perf-node-gather-daemonset-msf5w\" (UID: \"14829109-4102-4cc0-92cb-95c42f6b2373\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:43.182943 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.182827    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:43.464214 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.464128    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-749d795b56-bp9ph_65643f07-410d-45a0-a65a-b0cf8c547397/console/0.log"
Sep 09 12:56:43.481196 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.481161    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"]
Sep 09 12:56:43.483294 ip-10-0-139-173 kubenswrapper[2582]: W0909 12:56:43.483263    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod14829109_4102_4cc0_92cb_95c42f6b2373.slice/crio-2395c39fbcd713af4e41c6d890adfee70e62533ffcb9689f236e61fcf37b34fe WatchSource:0}: Error finding container 2395c39fbcd713af4e41c6d890adfee70e62533ffcb9689f236e61fcf37b34fe: Status 404 returned error can't find the container with id 2395c39fbcd713af4e41c6d890adfee70e62533ffcb9689f236e61fcf37b34fe
Sep 09 12:56:43.499478 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.499453    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-w66hv_1637635d-cc6c-490f-801d-f37a0b675865/download-server/0.log"
Sep 09 12:56:43.917572 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.917531    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w" event={"ID":"14829109-4102-4cc0-92cb-95c42f6b2373","Type":"ContainerStarted","Data":"0fa409614ed5deaf55e9e107387062b1cc6f4d0b5dd879490322121e3b78554e"}
Sep 09 12:56:43.918100 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.917592    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w" event={"ID":"14829109-4102-4cc0-92cb-95c42f6b2373","Type":"ContainerStarted","Data":"2395c39fbcd713af4e41c6d890adfee70e62533ffcb9689f236e61fcf37b34fe"}
Sep 09 12:56:43.918100 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.917730    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:43.935426 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:43.935363    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w" podStartSLOduration=1.935341929 podStartE2EDuration="1.935341929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:56:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:56:43.932587392 +0000 UTC m=+2223.629152558" watchObservedRunningTime="2026-09-09 12:56:43.935341929 +0000 UTC m=+2223.631907083"
Sep 09 12:56:45.401566 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:45.401536    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lq49k_72bee612-067e-4654-9095-41e974f65516/dns/0.log"
Sep 09 12:56:45.423038 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:45.423009    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lq49k_72bee612-067e-4654-9095-41e974f65516/kube-rbac-proxy/0.log"
Sep 09 12:56:45.451465 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:45.451434    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8zf7j_25a2a8b0-4d6b-4171-b470-7fd9b57a16e8/dns-node-resolver/0.log"
Sep 09 12:56:46.041824 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:46.041795    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-68f8668478-rg5qk_9cff25b0-468a-4261-bd8e-a5560517ce8e/registry/0.log"
Sep 09 12:56:46.062347 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:46.062324    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bjqck_ae8eed86-8b04-4c9a-ab62-4ea97a34849d/node-ca/0.log"
Sep 09 12:56:47.599385 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:47.599357    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5948bbb55b-hznrz_6946ad55-ed72-48f5-a38d-a722eac0e7f0/kube-auth-proxy/0.log"
Sep 09 12:56:47.629282 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:47.629254    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-msm2r_4efdd84f-7b2d-43a6-b028-5426f2b68203/istio-proxy/0.log"
Sep 09 12:56:47.641151 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:47.641111    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-msm2r_4efdd84f-7b2d-43a6-b028-5426f2b68203/istio-proxy/1.log"
Sep 09 12:56:47.728141 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:47.728114    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hclrp_fec7ad7e-69c0-4f61-9c45-4214164224b5/payload-processing/0.log"
Sep 09 12:56:48.660738 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:48.660704    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z8n7s_9e6412b5-162d-4538-8533-8806bbe0e8c9/serve-healthcheck-canary/0.log"
Sep 09 12:56:49.246904 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:49.246870    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gf8fx_8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4/kube-rbac-proxy/0.log"
Sep 09 12:56:49.270105 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:49.270082    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gf8fx_8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4/exporter/0.log"
Sep 09 12:56:49.293279 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:49.293255    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gf8fx_8ea0d14c-1bab-4d7d-b951-8e6f0b46c0e4/extractor/0.log"
Sep 09 12:56:49.940171 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:49.940136    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-msf5w"
Sep 09 12:56:51.488688 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:51.488666    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xk9r2_7d39ec25-d666-4150-b545-d3d6bf8c9417/manager/0.log"
Sep 09 12:56:51.721669 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:51.721641    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jn9jk_37efbe2e-4604-48f8-982f-babd48f9aba3/manager/0.log"
Sep 09 12:56:51.779415 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:51.779382    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-78bcb7bc6c-fhhml_10e3be91-1690-4dc1-ba02-4bdef218d5f7/manager/0.log"
Sep 09 12:56:52.159647 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:52.159613    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c5d49f968-x466m_9b86bae9-72d0-48c2-b3c2-91651d80a6d3/manager/0.log"
Sep 09 12:56:52.180483 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:52.180442    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c5d49f968-x466m_9b86bae9-72d0-48c2-b3c2-91651d80a6d3/copy-manifests/0.log"
Sep 09 12:56:53.664535 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:53.664507    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5948bbb55b-hznrz_6946ad55-ed72-48f5-a38d-a722eac0e7f0/kube-auth-proxy/0.log"
Sep 09 12:56:53.686704 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:53.686676    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-msm2r_4efdd84f-7b2d-43a6-b028-5426f2b68203/istio-proxy/0.log"
Sep 09 12:56:53.698464 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:53.698432    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-msm2r_4efdd84f-7b2d-43a6-b028-5426f2b68203/istio-proxy/1.log"
Sep 09 12:56:53.767700 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:53.767665    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hclrp_fec7ad7e-69c0-4f61-9c45-4214164224b5/payload-processing/0.log"
Sep 09 12:56:55.694557 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:55.694526    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5948bbb55b-hznrz_6946ad55-ed72-48f5-a38d-a722eac0e7f0/kube-auth-proxy/0.log"
Sep 09 12:56:55.716204 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:55.716170    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-msm2r_4efdd84f-7b2d-43a6-b028-5426f2b68203/istio-proxy/0.log"
Sep 09 12:56:55.727366 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:55.727345    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-msm2r_4efdd84f-7b2d-43a6-b028-5426f2b68203/istio-proxy/1.log"
Sep 09 12:56:55.800996 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:56:55.800966    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hclrp_fec7ad7e-69c0-4f61-9c45-4214164224b5/payload-processing/0.log"
Sep 09 12:57:00.001496 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:57:00.001471    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8tjr5_18511945-5b00-4ea7-8c23-598b3f3de208/kube-multus-additional-cni-plugins/0.log"
Sep 09 12:57:00.026205 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:57:00.026152    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8tjr5_18511945-5b00-4ea7-8c23-598b3f3de208/egress-router-binary-copy/0.log"
Sep 09 12:57:00.075592 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:57:00.075566    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8tjr5_18511945-5b00-4ea7-8c23-598b3f3de208/cni-plugins/0.log"
Sep 09 12:57:00.100675 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:57:00.100641    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8tjr5_18511945-5b00-4ea7-8c23-598b3f3de208/bond-cni-plugin/0.log"
Sep 09 12:57:00.123712 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:57:00.123677    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8tjr5_18511945-5b00-4ea7-8c23-598b3f3de208/routeoverride-cni/0.log"
Sep 09 12:57:00.147385 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:57:00.147358    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8tjr5_18511945-5b00-4ea7-8c23-598b3f3de208/whereabouts-cni-bincopy/0.log"
Sep 09 12:57:00.173095 ip-10-0-139-173 kubenswrapper[2582]: I0909 12:57:00.173069    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8tjr5_18511945-5b00-4ea7-8c23-598b3f3de208/whereabouts-cni/0.log"