Sep 14 01:30:40.413285 ip-10-0-132-11 systemd[1]: Starting Kubernetes Kubelet...
Sep 14 01:30:40.843296 ip-10-0-132-11 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 01:30:40.843296 ip-10-0-132-11 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 14 01:30:40.843296 ip-10-0-132-11 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 01:30:40.843296 ip-10-0-132-11 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 14 01:30:40.843296 ip-10-0-132-11 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 01:30:40.845404 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.845304    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 14 01:30:40.849802 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849779    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 01:30:40.849802 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849801    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849805    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849809    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849812    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849815    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849818    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849822    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849826    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849830    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849833    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849836    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849839    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849842    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849851    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849854    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849857    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849859    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849862    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849865    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849867    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 01:30:40.849877 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849870    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849872    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849875    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849878    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849881    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849884    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849887    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849891    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849894    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849899    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849903    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849926    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849930    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849933    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849935    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849938    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849941    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849943    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849946    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 01:30:40.850409 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849948    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849951    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849953    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849956    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849958    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849961    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849964    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849967    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849970    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849973    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849975    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849978    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849980    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849983    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849986    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849989    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849993    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849995    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.849998    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 01:30:40.850884 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850001    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850003    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850006    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850010    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850012    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850015    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850018    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850020    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850022    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850043    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850046    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850048    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850051    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850053    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850056    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850059    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850061    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850064    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850066    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850069    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 01:30:40.851368 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850071    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850074    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850077    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850080    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850082    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850085    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850088    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850091    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850094    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850097    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850099    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850102    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850105    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850108    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850111    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850113    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850116    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850119    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850122    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.850124    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 01:30:40.851852 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851686    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851695    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851699    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851702    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851705    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851709    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851712    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851714    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851718    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851720    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851723    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851726    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851728    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851731    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851735    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851737    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851740    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851742    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851745    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851749    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 01:30:40.852443 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851752    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851755    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851758    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851761    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851764    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851767    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851769    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851772    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851774    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851777    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851780    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851783    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851785    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851788    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851790    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851793    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851795    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851798    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851800    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851803    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 01:30:40.852974 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851805    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851807    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851810    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851812    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851816    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851819    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851821    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851826    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851828    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851831    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851834    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851838    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851840    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851843    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851845    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851848    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851853    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851856    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851860    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 01:30:40.853473 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851863    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851865    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851868    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851871    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851874    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851877    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851880    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851884    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851887    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851890    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851893    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851896    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851898    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851901    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851904    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851907    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851926    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851929    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851932    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 01:30:40.853975 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851934    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851937    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851940    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851943    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851946    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851949    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851953    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851956    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851959    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851961    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851965    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851967    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851970    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851972    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851975    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851979    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851983    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851986    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851988    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 01:30:40.854433 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851991    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.851993    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852419    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852425    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852429    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852432    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852434    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852437    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852439    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852443    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852446    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852449    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852452    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852455    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852458    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852461    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852464    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852467    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852470    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852473    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 01:30:40.854949 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852476    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852479    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852483    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852486    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852489    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852492    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852495    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852498    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852501    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852505    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852508    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852510    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852513    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852515    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852518    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852521    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852523    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852526    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852529    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 01:30:40.855450 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852531    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852535    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852538    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852541    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852543    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852546    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852548    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852551    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852553    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852556    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852559    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852562    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852565    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852567    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852570    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852572    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852575    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852577    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852580    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852582    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 01:30:40.855928 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852585    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852587    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852590    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852593    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852595    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852598    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852601    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852603    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852605    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852608    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852610    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852613    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852615    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852618    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852621    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852624    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852626    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852629    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852631    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852634    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 01:30:40.856456 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852636    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852638    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852642    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852644    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852647    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852649    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852652    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852654    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852657    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852660    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852663    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852667    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852670    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852672    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852675    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852677    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852680    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852682    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852685    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 01:30:40.856957 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852688    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852692    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.852695    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852766    2581 flags.go:64] FLAG: --address="0.0.0.0"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852797    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852805    2581 flags.go:64] FLAG: --anonymous-auth="true"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852810    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852814    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852818    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852822    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852827    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852830    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852833    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852837    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852840    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852843    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852848    2581 flags.go:64] FLAG: --cgroup-root=""
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852851    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852854    2581 flags.go:64] FLAG: --client-ca-file=""
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852857    2581 flags.go:64] FLAG: --cloud-provider="external"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852860    2581 flags.go:64] FLAG: --cluster-dns="[]"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852864    2581 flags.go:64] FLAG: --cluster-domain=""
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852867    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852871    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 14 01:30:40.857429 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852874    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852877    2581 flags.go:64] FLAG: --container-log-max-files="5"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852881    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852884    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852887    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852890    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852893    2581 flags.go:64] FLAG: --contention-profiling="false"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852896    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852899    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852903    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852905    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852922    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852925    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852929    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852932    2581 flags.go:64] FLAG: --enable-load-reader="false"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852935    2581 flags.go:64] FLAG: --enable-server="true"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852938    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852942    2581 flags.go:64] FLAG: --event-burst="100"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852945    2581 flags.go:64] FLAG: --event-qps="50"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852949    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852952    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852955    2581 flags.go:64] FLAG: --eviction-hard=""
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852958    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852962    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852965    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 14 01:30:40.858076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852969    2581 flags.go:64] FLAG: --eviction-soft=""
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852972    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852975    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852978    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852981    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852984    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852987    2581 flags.go:64] FLAG: --fail-swap-on="true"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852990    2581 flags.go:64] FLAG: --feature-gates=""
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852994    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.852998    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853001    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853004    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853007    2581 flags.go:64] FLAG: --healthz-port="10248"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853010    2581 flags.go:64] FLAG: --help="false"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853013    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-132-11.ec2.internal"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853017    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853020    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853023    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853026    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853030    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853033    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853036    2581 flags.go:64] FLAG: --image-service-endpoint=""
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853039    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 14 01:30:40.858678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853042    2581 flags.go:64] FLAG: --kube-api-burst="100"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853045    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853048    2581 flags.go:64] FLAG: --kube-api-qps="50"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853051    2581 flags.go:64] FLAG: --kube-reserved=""
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853054    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853057    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853060    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853063    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853066    2581 flags.go:64] FLAG: --lock-file=""
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853069    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853072    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853076    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853081    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853084    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853086    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853089    2581 flags.go:64] FLAG: --logging-format="text"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853092    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853095    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853098    2581 flags.go:64] FLAG: --manifest-url=""
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853102    2581 flags.go:64] FLAG: --manifest-url-header=""
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853109    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853112    2581 flags.go:64] FLAG: --max-open-files="1000000"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853116    2581 flags.go:64] FLAG: --max-pods="110"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853119    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853122    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 14 01:30:40.859279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853125    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853128    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853131    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853134    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853137    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853145    2581 flags.go:64] FLAG: --node-status-max-images="50"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853150    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853153    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853156    2581 flags.go:64] FLAG: --pod-cidr=""
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853161    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853168    2581 flags.go:64] FLAG: --pod-manifest-path=""
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853171    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853174    2581 flags.go:64] FLAG: --pods-per-core="0"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853177    2581 flags.go:64] FLAG: --port="10250"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853180    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853183    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0565fa9ef7e5098e3"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853186    2581 flags.go:64] FLAG: --qos-reserved=""
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853189    2581 flags.go:64] FLAG: --read-only-port="10255"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853193    2581 flags.go:64] FLAG: --register-node="true"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853196    2581 flags.go:64] FLAG: --register-with-taints=""
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853200    2581 flags.go:64] FLAG: --registry-burst="10"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853203    2581 flags.go:64] FLAG: --registry-qps="5"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853205    2581 flags.go:64] FLAG: --reserved-cpus=""
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853208    2581 flags.go:64] FLAG: --reserved-memory=""
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853212    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 14 01:30:40.859900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853215    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853218    2581 flags.go:64] FLAG: --rotate-certificates="false"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853221    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853224    2581 flags.go:64] FLAG: --runonce="false"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853227    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853230    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853233    2581 flags.go:64] FLAG: --seccomp-default="false"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853236    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853238    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853242    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853245    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853248    2581 flags.go:64] FLAG: --storage-driver-password="root"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853251    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853256    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853259    2581 flags.go:64] FLAG: --storage-driver-user="root"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853262    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853264    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853269    2581 flags.go:64] FLAG: --system-cgroups=""
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853272    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853277    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853280    2581 flags.go:64] FLAG: --tls-cert-file=""
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853283    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853288    2581 flags.go:64] FLAG: --tls-min-version=""
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853291    2581 flags.go:64] FLAG: --tls-private-key-file=""
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853294    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 14 01:30:40.860535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853297    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853300    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853303    2581 flags.go:64] FLAG: --v="2"
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853307    2581 flags.go:64] FLAG: --version="false"
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853312    2581 flags.go:64] FLAG: --vmodule=""
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853316    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.853319    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853444    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853448    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853451    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853456    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853459    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853462    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853465    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853468    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853471    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853474    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853476    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853479    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853482    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853487    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853490    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 01:30:40.861190 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853493    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853496    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853500    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853502    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853505    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853508    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853511    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853513    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853516    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853519    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853521    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853525    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853528    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853531    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853533    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853536    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853538    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853541    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853545    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853548    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 01:30:40.861930 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853551    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853554    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853557    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853560    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853562    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853565    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853567    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853570    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853572    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853575    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853579    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853582    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853585    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853587    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853592    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853595    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853597    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853600    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853602    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 01:30:40.862696 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853605    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853608    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853610    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853613    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853615    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853619    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853622    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853625    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853627    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853630    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853632    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853635    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853638    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853640    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853643    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853647    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853650    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853653    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853657    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 01:30:40.863213 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853661    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853665    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853668    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853672    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853677    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853680    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853683    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853685    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853691    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853694    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853696    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853699    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853701    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853704    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853707    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853710    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853712    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853715    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853717    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 01:30:40.863698 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853720    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 01:30:40.864198 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853724    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 01:30:40.864198 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853727    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 01:30:40.864198 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853730    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 01:30:40.864198 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853732    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 01:30:40.864198 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853735    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 01:30:40.864198 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.853738    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 01:30:40.864198 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.854498    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 01:30:40.864198 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.863683    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 14 01:30:40.864198 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.863701    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 14 01:30:40.864516 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.864504    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 14 01:30:40.864545 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.864517    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 14 01:30:40.864589 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864580    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 01:30:40.864589 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864589    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864592    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864596    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864600    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864603    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864606    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864609    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864613    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864616    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864618    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864621    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864624    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864626    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864630    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864632    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864635    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864638    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864640    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864643    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864646    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 01:30:40.864646 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864649    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864653    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864656    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864659    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864662    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864664    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864667    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864670    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864672    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864675    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864678    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864680    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864684    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864686    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864689    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864691    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864694    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864697    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864699    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864702    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 01:30:40.865275 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864705    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864707    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864710    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864712    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864715    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864717    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864720    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864723    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864725    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864728    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864730    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864733    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864736    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864739    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864743    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864746    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864748    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864751    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864755    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 01:30:40.865806 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864760    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864763    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864765    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864768    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864771    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864775    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864778    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864782    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864785    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864788    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864790    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864793    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864795    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864798    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864819    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864823    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864827    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864830    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864833    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 01:30:40.866303 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864837    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864840    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864843    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864846    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864848    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864851    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864854    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864858    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864861    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864864    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864866    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864869    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864873    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864876    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864878    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864882    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864885    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864888    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864891    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 01:30:40.866766 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.864894    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 01:30:40.867268 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.864899    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 01:30:40.867268 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865037    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 01:30:40.867268 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865043    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 01:30:40.867268 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865046    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 01:30:40.867268 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865049    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 01:30:40.867268 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865052    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 01:30:40.867268 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865055    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 01:30:40.867268 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865058    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 01:30:40.867268 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865062    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 01:30:40.867268 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865064    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 01:30:40.867268 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865067    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 01:30:40.867268 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865070    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 01:30:40.867268 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865074    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 01:30:40.867268 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865079    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 01:30:40.867268 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865082    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865084    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865087    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865090    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865092    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865095    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865098    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865101    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865104    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865107    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865110    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865112    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865115    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865118    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865120    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865123    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865126    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865129    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865131    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865134    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 01:30:40.867647 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865138    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865141    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865144    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865146    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865149    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865152    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865154    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865157    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865160    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865162    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865165    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865167    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865171    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865174    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865176    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865179    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865181    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865184    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865187    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865190    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 01:30:40.868141 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865192    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865195    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865198    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865200    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865203    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865206    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865208    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865211    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865213    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865216    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865219    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865222    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865224    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865228    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865232    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865235    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865238    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865241    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865244    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 01:30:40.868627 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865247    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865250    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865253    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865255    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865258    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865260    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865263    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865266    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865269    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865271    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865274    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865277    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865280    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865283    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865285    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865288    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865291    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865294    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865297    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865299    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 01:30:40.869106 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865302    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 01:30:40.869575 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865304    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 01:30:40.869575 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865307    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 01:30:40.869575 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865310    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 01:30:40.869575 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865312    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 01:30:40.869575 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865315    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 01:30:40.869575 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:40.865318    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 01:30:40.869575 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.865323    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 01:30:40.869575 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.865474    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 14 01:30:40.870675 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.870658    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 14 01:30:40.871782 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.871769    2581 server.go:1019] "Starting client certificate rotation"
Sep 14 01:30:40.871902 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.871881    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 01:30:40.871967 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.871944    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 01:30:40.874533 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.874514    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 01:30:40.875723 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.875703    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 01:30:40.888674 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.888652    2581 log.go:25] "Validated CRI v1 runtime API"
Sep 14 01:30:40.893361 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.893340    2581 log.go:25] "Validated CRI v1 image API"
Sep 14 01:30:40.894249 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.894233    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 14 01:30:40.897631 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.897611    2581 fs.go:135] Filesystem UUIDs: map[0621cfb7-111d-46a3-8778-e60119a9c2a5:/dev/nvme0n1p3 611c942f-c111-4e00-8fb9-e5d36e6e27fc:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 14 01:30:40.897678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.897631    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 14 01:30:40.904605 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.904475    2581 manager.go:216] Machine: {Timestamp:2026-09-14 01:30:40.903033452 +0000 UTC m=+0.383484263 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3115280 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec276f56e4eb96cfb4cd00045c64324e SystemUUID:ec276f56-e4eb-96cf-b4cd-00045c64324e BootID:3570a2d7-5358-4f2a-83c7-bded58fe5262 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:eb:61:d6:37 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:eb:61:d6:37 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:52:4e:2f:eb:c6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 14 01:30:40.904605 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.904592    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 14 01:30:40.904792 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.904683    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 01:30:40.904792 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.904695    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 14 01:30:40.908506 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.908481    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 14 01:30:40.908655 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.908509    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-11.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 14 01:30:40.909379 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.909368    2581 topology_manager.go:138] "Creating topology manager with none policy"
Sep 14 01:30:40.909411 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.909382    2581 container_manager_linux.go:309] "Creating device plugin manager"
Sep 14 01:30:40.909411 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.909390    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 01:30:40.909411 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.909404    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 01:30:40.911395 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.911383    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 14 01:30:40.916055 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.916041    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 01:30:40.916173 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.916164    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 14 01:30:40.916205 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.916194    2581 kubelet.go:486] "Attempting to sync node with API server"
Sep 14 01:30:40.916205 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.916203    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 14 01:30:40.916262 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.916214    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 14 01:30:40.916262 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.916226    2581 kubelet.go:398] "Adding apiserver pod source"
Sep 14 01:30:40.916262 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.916234    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 14 01:30:40.916754 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.916735    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 01:30:40.916840 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.916764    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 01:30:40.919256 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.919234    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 14 01:30:40.922247 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.922232    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 14 01:30:40.922321 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.922263    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 14 01:30:40.925062 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.925047    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 14 01:30:40.925141 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.925071    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 14 01:30:40.925141 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.925078    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 14 01:30:40.925141 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.925084    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 14 01:30:40.925141 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.925092    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 14 01:30:40.925141 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.925098    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 14 01:30:40.925141 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.925113    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 14 01:30:40.925141 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.925118    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 14 01:30:40.925141 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.925125    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 14 01:30:40.925141 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.925131    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 14 01:30:40.925141 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.925140    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 14 01:30:40.925141 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.925145    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 14 01:30:40.926715 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:40.926686    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 14 01:30:40.926775 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:40.926681    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 14 01:30:40.926946 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.926935    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 14 01:30:40.926983 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.926948    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 14 01:30:40.930841 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.930830    2581 server.go:1268] "Started kubelet"
Sep 14 01:30:40.930963 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.930927    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 14 01:30:40.931029 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.930979    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 14 01:30:40.931077 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.931056    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 14 01:30:40.931827 ip-10-0-132-11 systemd[1]: Started Kubernetes Kubelet.
Sep 14 01:30:40.932819 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.932803    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 14 01:30:40.933099 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.933081    2581 server.go:310] "Adding debug handlers to kubelet server"
Sep 14 01:30:40.936442 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:40.936405    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 14 01:30:40.938842 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.938824    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-11.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 14 01:30:40.939881 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:40.938721    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-11.ec2.internal.18d50bd60db96768  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-11.ec2.internal,UID:ip-10-0-132-11.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-11.ec2.internal,},FirstTimestamp:2026-09-14 01:30:40.930801512 +0000 UTC m=+0.411252323,LastTimestamp:2026-09-14 01:30:40.930801512 +0000 UTC m=+0.411252323,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-11.ec2.internal,}"
Sep 14 01:30:40.941241 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.941225    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 14 01:30:40.941944 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.941903    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 14 01:30:40.942028 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:40.942020    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:40.942639 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.942620    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 14 01:30:40.942639 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.942639    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 14 01:30:40.942771 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.942621    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 14 01:30:40.942843 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.942774    2581 reconstruct.go:97] "Volume reconstruction finished"
Sep 14 01:30:40.942843 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.942779    2581 reconciler.go:29] "Reconciler: start to sync state"
Sep 14 01:30:40.942900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.942866    2581 factory.go:55] Registering systemd factory
Sep 14 01:30:40.942900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.942882    2581 factory.go:223] Registration of the systemd container factory successfully
Sep 14 01:30:40.943166 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.943150    2581 factory.go:153] Registering CRI-O factory
Sep 14 01:30:40.943223 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.943169    2581 factory.go:223] Registration of the crio container factory successfully
Sep 14 01:30:40.943270 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.943226    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 14 01:30:40.943270 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.943246    2581 factory.go:103] Registering Raw factory
Sep 14 01:30:40.943270 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.943261    2581 manager.go:1196] Started watching for new ooms in manager
Sep 14 01:30:40.943576 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:40.943555    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-11.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 14 01:30:40.943635 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:40.943615    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 14 01:30:40.943686 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.943646    2581 manager.go:349] Starting recovery of all containers
Sep 14 01:30:40.945891 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.945862    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t7cl9"
Sep 14 01:30:40.950826 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.950799    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t7cl9"
Sep 14 01:30:40.953187 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.953148    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 14 01:30:40.956095 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.955617    2581 manager.go:354] Recovery completed
Sep 14 01:30:40.957387 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:40.957352    2581 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 14 01:30:40.960650 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.960636    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 01:30:40.962742 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.962720    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 01:30:40.962821 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.962761    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 01:30:40.962821 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.962775    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientPID"
Sep 14 01:30:40.963420 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.963405    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 14 01:30:40.963420 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.963419    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 14 01:30:40.963554 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.963437    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 01:30:40.963554 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.963450    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 01:30:40.964162 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.964143    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 01:30:40.964220 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.964172    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 01:30:40.964220 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.964182    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientPID"
Sep 14 01:30:40.965625 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.965613    2581 policy_none.go:49] "None policy: Start"
Sep 14 01:30:40.965670 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.965633    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 14 01:30:40.965670 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.965646    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 14 01:30:40.967723 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.967710    2581 policy_none.go:47] "Start"
Sep 14 01:30:40.992611 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.992590    2581 manager.go:337] "Starting Device Plugin manager"
Sep 14 01:30:40.992731 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:40.992647    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 14 01:30:40.992731 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.992661    2581 server.go:87] "Starting device plugin registration server"
Sep 14 01:30:40.993064 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.993049    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 14 01:30:40.993139 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.993064    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 14 01:30:40.993199 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.993186    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 14 01:30:40.993337 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.993261    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 14 01:30:40.993337 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:40.993273    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 14 01:30:40.993955 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:40.993871    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 14 01:30:40.993955 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:40.993953    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:41.046997 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.046963    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 14 01:30:41.049183 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.047026    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 14 01:30:41.049183 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.047053    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 14 01:30:41.049183 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:41.047172    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 14 01:30:41.049348 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.049334    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 01:30:41.094090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.093968    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 01:30:41.096171 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.096147    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 01:30:41.096171 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.096177    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 01:30:41.096306 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.096189    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientPID"
Sep 14 01:30:41.096306 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.096212    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.105181 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.105152    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.105181 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:41.105181    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-11.ec2.internal\": node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:41.126359 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:41.126333    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:41.147324 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.147283    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal"]
Sep 14 01:30:41.147491 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.147377    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 01:30:41.148787 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.148767    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 01:30:41.148869 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.148801    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 01:30:41.148869 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.148812    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientPID"
Sep 14 01:30:41.151138 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.151125    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 01:30:41.151250 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.151236    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.151308 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.151270    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 01:30:41.151900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.151881    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 01:30:41.152027 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.151924    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 01:30:41.152027 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.151938    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientPID"
Sep 14 01:30:41.152324 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.152310    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 01:30:41.152381 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.152337    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 01:30:41.152381 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.152353    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientPID"
Sep 14 01:30:41.154600 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.154585    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.154668 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.154616    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 01:30:41.155347 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.155330    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 01:30:41.155448 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.155359    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 01:30:41.155448 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.155370    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeHasSufficientPID"
Sep 14 01:30:41.178724 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:41.178693    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-11.ec2.internal\" not found" node="ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.182088 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:41.182068    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-11.ec2.internal\" not found" node="ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.227460 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:41.227424    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:41.244273 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.244248    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e4be2f7c78788b0bf274c1ec5896ab29-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal\" (UID: \"e4be2f7c78788b0bf274c1ec5896ab29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.244417 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.244279    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4be2f7c78788b0bf274c1ec5896ab29-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal\" (UID: \"e4be2f7c78788b0bf274c1ec5896ab29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.244417 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.244297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fd5c0f6fc359fb985fe321e9822e84e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-11.ec2.internal\" (UID: \"9fd5c0f6fc359fb985fe321e9822e84e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.327759 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:41.327717    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:41.345122 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.345069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e4be2f7c78788b0bf274c1ec5896ab29-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal\" (UID: \"e4be2f7c78788b0bf274c1ec5896ab29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.345122 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.345095    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4be2f7c78788b0bf274c1ec5896ab29-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal\" (UID: \"e4be2f7c78788b0bf274c1ec5896ab29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.345122 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.345112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fd5c0f6fc359fb985fe321e9822e84e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-11.ec2.internal\" (UID: \"9fd5c0f6fc359fb985fe321e9822e84e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.345248 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.345141    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fd5c0f6fc359fb985fe321e9822e84e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-11.ec2.internal\" (UID: \"9fd5c0f6fc359fb985fe321e9822e84e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.346792 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.345371    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e4be2f7c78788b0bf274c1ec5896ab29-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal\" (UID: \"e4be2f7c78788b0bf274c1ec5896ab29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.346792 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.345167    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4be2f7c78788b0bf274c1ec5896ab29-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal\" (UID: \"e4be2f7c78788b0bf274c1ec5896ab29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.428576 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:41.428527    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:41.481087 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.481043    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.486275 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.486208    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal"
Sep 14 01:30:41.529411 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:41.529369    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:41.634178 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:41.634074    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:41.735907 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:41.735862    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:41.839570 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:41.839519    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:41.872462 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.872422    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 14 01:30:41.872975 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.872604    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 01:30:41.939823 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:41.939720    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:41.959268 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.959225    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-13 01:25:41 +0000 UTC" deadline="2028-02-28 04:24:29.781101012 +0000 UTC"
Sep 14 01:30:41.959268 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:41.959266    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12770h53m47.821839189s"
Sep 14 01:30:42.038488 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.038459    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 01:30:42.040098 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:42.040065    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:42.128483 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.128438    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 01:30:42.141214 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:42.141175    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:42.242490 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:42.242449    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:42.262156 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:42.262120    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4be2f7c78788b0bf274c1ec5896ab29.slice/crio-e1a8619aaad8fa2bf9546391b5dc5e3ad4785aea02503c47a28e4d2bb3af9063 WatchSource:0}: Error finding container e1a8619aaad8fa2bf9546391b5dc5e3ad4785aea02503c47a28e4d2bb3af9063: Status 404 returned error can't find the container with id e1a8619aaad8fa2bf9546391b5dc5e3ad4785aea02503c47a28e4d2bb3af9063
Sep 14 01:30:42.262379 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:42.262353    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd5c0f6fc359fb985fe321e9822e84e.slice/crio-a7eb1d966b2878351abc9609a6799525ed125f6c771ccfc57daaf2e14762f53b WatchSource:0}: Error finding container a7eb1d966b2878351abc9609a6799525ed125f6c771ccfc57daaf2e14762f53b: Status 404 returned error can't find the container with id a7eb1d966b2878351abc9609a6799525ed125f6c771ccfc57daaf2e14762f53b
Sep 14 01:30:42.266789 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.266771    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 01:30:42.343120 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:42.343067    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:42.444076 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:42.444036    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-11.ec2.internal\" not found"
Sep 14 01:30:42.481756 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.481730    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 01:30:42.542364 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.542275    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal"
Sep 14 01:30:42.555360 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.555334    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 01:30:42.556317 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.556301    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal"
Sep 14 01:30:42.563702 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.563684    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 01:30:42.919057 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.918963    2581 apiserver.go:52] "Watching apiserver"
Sep 14 01:30:42.923488 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.923460    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 14 01:30:42.923844 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.923819    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-gf7k5","kube-system/global-pull-secret-syncer-dw759","kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2","openshift-multus/multus-additional-cni-plugins-ktprp","openshift-multus/multus-fcn4g","openshift-multus/network-metrics-daemon-fv6nk","openshift-network-diagnostics/network-check-target-gg9cz","openshift-network-operator/iptables-alerter-hhpxs","kube-system/konnectivity-agent-wb7hr","openshift-cluster-node-tuning-operator/tuned-x5mcp","openshift-dns/node-resolver-bzb4b","openshift-image-registry/node-ca-rlnvn"]
Sep 14 01:30:42.928026 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.928007    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hhpxs"
Sep 14 01:30:42.930218 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.930193    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 14 01:30:42.930356 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.930225    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ckrgq\""
Sep 14 01:30:42.930356 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.930243    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 14 01:30:42.930356 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.930278    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 14 01:30:42.930356 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.930335    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bzb4b"
Sep 14 01:30:42.932357 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.932332    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6sf82\""
Sep 14 01:30:42.932588 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.932571    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 14 01:30:42.932679 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.932652    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 14 01:30:42.933299 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.933275    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:42.933386 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:42.933357    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:30:42.935514 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.935498    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rlnvn"
Sep 14 01:30:42.938181 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.938163    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 14 01:30:42.939084 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.939063    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-77946\""
Sep 14 01:30:42.939304 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.938234    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:42.939500 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.938919    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 14 01:30:42.939500 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.939457    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 14 01:30:42.942121 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.941809    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 14 01:30:42.942121 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.942055    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 14 01:30:42.942308 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.942162    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 14 01:30:42.942308 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.942209    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 14 01:30:42.942308 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.942240    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4xbn9\""
Sep 14 01:30:42.942665 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.942642    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:42.944785 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.944738    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 14 01:30:42.944785 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.944758    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 14 01:30:42.945069 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.945048    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-c6w2w\""
Sep 14 01:30:42.945246 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.945220    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 14 01:30:42.945354 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.945227    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 14 01:30:42.945354 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.945232    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 14 01:30:42.945706 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.945687    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:42.945799 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:42.945759    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:30:42.948208 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.948186    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:42.950156 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.950132    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7chpb\""
Sep 14 01:30:42.950639 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.950511    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 14 01:30:42.951077 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.951055    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:42.953107 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.953084    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qwfkq\""
Sep 14 01:30:42.953205 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.953095    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 14 01:30:42.953832 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.953794    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 14 01:30:42.953914 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.953833    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 14 01:30:42.953914 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.953892    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 14 01:30:42.954177 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954049    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 14 01:30:42.954177 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954068    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 14 01:30:42.954455 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954426    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-etc-selinux\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954471    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4e0a5004-422d-425e-954f-1d2bcba81a37-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954502    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7cc6e131-076a-4f42-bdc9-874e49b5b938-dbus\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954552    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f00644fe-4d6e-49e3-a485-eb4029ad78ad-host\") pod \"node-ca-rlnvn\" (UID: \"f00644fe-4d6e-49e3-a485-eb4029ad78ad\") " pod="openshift-image-registry/node-ca-rlnvn"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954580    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954602    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f00644fe-4d6e-49e3-a485-eb4029ad78ad-serviceca\") pod \"node-ca-rlnvn\" (UID: \"f00644fe-4d6e-49e3-a485-eb4029ad78ad\") " pod="openshift-image-registry/node-ca-rlnvn"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954638    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xkdz\" (UniqueName: \"kubernetes.io/projected/33097456-38ba-4136-8f5c-b87cb1707501-kube-api-access-2xkdz\") pod \"iptables-alerter-hhpxs\" (UID: \"33097456-38ba-4136-8f5c-b87cb1707501\") " pod="openshift-network-operator/iptables-alerter-hhpxs"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954675    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e0a5004-422d-425e-954f-1d2bcba81a37-cnibin\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954702    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e0a5004-422d-425e-954f-1d2bcba81a37-system-cni-dir\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:42.954668    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954752    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j6g8\" (UniqueName: \"kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8\") pod \"network-check-target-gg9cz\" (UID: \"4ab6aa67-3f53-4c3f-b668-cbeb98b4010e\") " pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954780    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4ab20462-366d-4e07-a302-25be0aef9dd9-hosts-file\") pod \"node-resolver-bzb4b\" (UID: \"4ab20462-366d-4e07-a302-25be0aef9dd9\") " pod="openshift-dns/node-resolver-bzb4b"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954824    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33097456-38ba-4136-8f5c-b87cb1707501-host-slash\") pod \"iptables-alerter-hhpxs\" (UID: \"33097456-38ba-4136-8f5c-b87cb1707501\") " pod="openshift-network-operator/iptables-alerter-hhpxs"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954875    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-sys-fs\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954908    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.954950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7cc6e131-076a-4f42-bdc9-874e49b5b938-kubelet-config\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:42.955390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.955018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmhdr\" (UniqueName: \"kubernetes.io/projected/375beafc-69ac-49ff-bf02-5dd15f9d1692-kube-api-access-gmhdr\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:42.956150 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.955051    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrdt4\" (UniqueName: \"kubernetes.io/projected/4e0a5004-422d-425e-954f-1d2bcba81a37-kube-api-access-nrdt4\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:42.956150 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.955077    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ab20462-366d-4e07-a302-25be0aef9dd9-tmp-dir\") pod \"node-resolver-bzb4b\" (UID: \"4ab20462-366d-4e07-a302-25be0aef9dd9\") " pod="openshift-dns/node-resolver-bzb4b"
Sep 14 01:30:42.956150 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.955098    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-socket-dir\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:42.956150 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.955114    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e0a5004-422d-425e-954f-1d2bcba81a37-cni-binary-copy\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:42.956150 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.955137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4e0a5004-422d-425e-954f-1d2bcba81a37-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:42.956150 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.955275    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:42.956150 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.955308    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/33097456-38ba-4136-8f5c-b87cb1707501-iptables-alerter-script\") pod \"iptables-alerter-hhpxs\" (UID: \"33097456-38ba-4136-8f5c-b87cb1707501\") " pod="openshift-network-operator/iptables-alerter-hhpxs"
Sep 14 01:30:42.956150 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.955375    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e0a5004-422d-425e-954f-1d2bcba81a37-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:42.956150 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.955402    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh97l\" (UniqueName: \"kubernetes.io/projected/4ab20462-366d-4e07-a302-25be0aef9dd9-kube-api-access-qh97l\") pod \"node-resolver-bzb4b\" (UID: \"4ab20462-366d-4e07-a302-25be0aef9dd9\") " pod="openshift-dns/node-resolver-bzb4b"
Sep 14 01:30:42.956150 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.955428    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fsk8\" (UniqueName: \"kubernetes.io/projected/f00644fe-4d6e-49e3-a485-eb4029ad78ad-kube-api-access-4fsk8\") pod \"node-ca-rlnvn\" (UID: \"f00644fe-4d6e-49e3-a485-eb4029ad78ad\") " pod="openshift-image-registry/node-ca-rlnvn"
Sep 14 01:30:42.956150 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.955453    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-registration-dir\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:42.956150 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.955473    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-device-dir\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:42.956604 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.956150    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 01:30:42.957004 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.956980    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wb7hr"
Sep 14 01:30:42.959079 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.958877    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 14 01:30:42.959079 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.958909    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 14 01:30:42.959079 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.959028    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t7znq\""
Sep 14 01:30:42.959546 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.959530    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:42.961592 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.961572    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2zwzl\""
Sep 14 01:30:42.961695 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.961664    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 14 01:30:42.961798 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.961577    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 14 01:30:42.978872 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.978848    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-sc4rs"
Sep 14 01:30:42.987098 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:42.987072    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-sc4rs"
Sep 14 01:30:43.045292 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.045257    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 14 01:30:43.054852 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.054788    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal" event={"ID":"9fd5c0f6fc359fb985fe321e9822e84e","Type":"ContainerStarted","Data":"a7eb1d966b2878351abc9609a6799525ed125f6c771ccfc57daaf2e14762f53b"}
Sep 14 01:30:43.055738 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.055699    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dcc0a7d3-0591-4558-899a-0950f6920814-ovn-node-metrics-cert\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.055885 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.055749    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4e0a5004-422d-425e-954f-1d2bcba81a37-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:43.055885 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.055776    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:43.055885 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.055807    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-cnibin\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.055885 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.055831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-var-lib-kubelet\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.056172 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.055888    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-modprobe-d\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.056172 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.055934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-var-lib-kubelet\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.056172 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.055993    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-tmp\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.056172 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e0a5004-422d-425e-954f-1d2bcba81a37-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:43.056172 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056054    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh97l\" (UniqueName: \"kubernetes.io/projected/4ab20462-366d-4e07-a302-25be0aef9dd9-kube-api-access-qh97l\") pod \"node-resolver-bzb4b\" (UID: \"4ab20462-366d-4e07-a302-25be0aef9dd9\") " pod="openshift-dns/node-resolver-bzb4b"
Sep 14 01:30:43.056172 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056079    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-sysconfig\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.056172 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056102    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-sysctl-conf\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.056172 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056128    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-tuned\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.056172 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056152    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5b8p\" (UniqueName: \"kubernetes.io/projected/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-kube-api-access-j5b8p\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056178    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-device-dir\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7cc6e131-076a-4f42-bdc9-874e49b5b938-dbus\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f00644fe-4d6e-49e3-a485-eb4029ad78ad-host\") pod \"node-ca-rlnvn\" (UID: \"f00644fe-4d6e-49e3-a485-eb4029ad78ad\") " pod="openshift-image-registry/node-ca-rlnvn"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056258    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-run-netns\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056284    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-system-cni-dir\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056308    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-run-netns\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056333    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-hostroot\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056359    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-run-ovn-kubernetes\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056384    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbpcs\" (UniqueName: \"kubernetes.io/projected/dcc0a7d3-0591-4558-899a-0950f6920814-kube-api-access-sbpcs\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056409    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j6g8\" (UniqueName: \"kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8\") pod \"network-check-target-gg9cz\" (UID: \"4ab6aa67-3f53-4c3f-b668-cbeb98b4010e\") " pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056435    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-node-log\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056460    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-log-socket\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056475    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f00644fe-4d6e-49e3-a485-eb4029ad78ad-host\") pod \"node-ca-rlnvn\" (UID: \"f00644fe-4d6e-49e3-a485-eb4029ad78ad\") " pod="openshift-image-registry/node-ca-rlnvn"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056486    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33097456-38ba-4136-8f5c-b87cb1707501-host-slash\") pod \"iptables-alerter-hhpxs\" (UID: \"33097456-38ba-4136-8f5c-b87cb1707501\") " pod="openshift-network-operator/iptables-alerter-hhpxs"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056512    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-sys-fs\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056536    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-kubelet\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.056651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056568    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-systemd-units\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056594    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-cni-bin\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dcc0a7d3-0591-4558-899a-0950f6920814-ovnkube-script-lib\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7cc6e131-076a-4f42-bdc9-874e49b5b938-kubelet-config\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056673    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-run-openvswitch\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056707    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dcc0a7d3-0591-4558-899a-0950f6920814-ovnkube-config\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056743    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-kubernetes\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056769    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-systemd\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056778    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7cc6e131-076a-4f42-bdc9-874e49b5b938-dbus\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056796    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-lib-modules\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmhdr\" (UniqueName: \"kubernetes.io/projected/375beafc-69ac-49ff-bf02-5dd15f9d1692-kube-api-access-gmhdr\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056854    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrdt4\" (UniqueName: \"kubernetes.io/projected/4e0a5004-422d-425e-954f-1d2bcba81a37-kube-api-access-nrdt4\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056883    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/86a1cb60-daa6-4531-b896-ea13966bd97c-konnectivity-ca\") pod \"konnectivity-agent-wb7hr\" (UID: \"86a1cb60-daa6-4531-b896-ea13966bd97c\") " pod="kube-system/konnectivity-agent-wb7hr"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056908    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-cni-binary-copy\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056935    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-socket-dir\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e0a5004-422d-425e-954f-1d2bcba81a37-cni-binary-copy\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:43.057598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057017    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-run-multus-certs\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057040    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-cni-netd\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057062    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dcc0a7d3-0591-4558-899a-0950f6920814-env-overrides\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057087    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057117    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h5zv\" (UniqueName: \"kubernetes.io/projected/ff2b27ad-c198-489c-9bb8-befeea447a64-kube-api-access-8h5zv\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/33097456-38ba-4136-8f5c-b87cb1707501-iptables-alerter-script\") pod \"iptables-alerter-hhpxs\" (UID: \"33097456-38ba-4136-8f5c-b87cb1707501\") " pod="openshift-network-operator/iptables-alerter-hhpxs"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fsk8\" (UniqueName: \"kubernetes.io/projected/f00644fe-4d6e-49e3-a485-eb4029ad78ad-kube-api-access-4fsk8\") pod \"node-ca-rlnvn\" (UID: \"f00644fe-4d6e-49e3-a485-eb4029ad78ad\") " pod="openshift-image-registry/node-ca-rlnvn"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057196    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-etc-openvswitch\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057225    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057252    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-host\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057294    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-registration-dir\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057326    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-etc-selinux\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057378    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4e0a5004-422d-425e-954f-1d2bcba81a37-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057406    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f00644fe-4d6e-49e3-a485-eb4029ad78ad-serviceca\") pod \"node-ca-rlnvn\" (UID: \"f00644fe-4d6e-49e3-a485-eb4029ad78ad\") " pod="openshift-image-registry/node-ca-rlnvn"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057434    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-run-ovn\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xkdz\" (UniqueName: \"kubernetes.io/projected/33097456-38ba-4136-8f5c-b87cb1707501-kube-api-access-2xkdz\") pod \"iptables-alerter-hhpxs\" (UID: \"33097456-38ba-4136-8f5c-b87cb1707501\") " pod="openshift-network-operator/iptables-alerter-hhpxs"
Sep 14 01:30:43.058444 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057495    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e0a5004-422d-425e-954f-1d2bcba81a37-cnibin\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057524    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-multus-daemon-config\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057525    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e0a5004-422d-425e-954f-1d2bcba81a37-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057574    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-slash\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-run\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e0a5004-422d-425e-954f-1d2bcba81a37-system-cni-dir\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057675    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4ab20462-366d-4e07-a302-25be0aef9dd9-hosts-file\") pod \"node-resolver-bzb4b\" (UID: \"4ab20462-366d-4e07-a302-25be0aef9dd9\") " pod="openshift-dns/node-resolver-bzb4b"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057730    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e0a5004-422d-425e-954f-1d2bcba81a37-system-cni-dir\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057744    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-run-k8s-cni-cncf-io\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057786    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4e0a5004-422d-425e-954f-1d2bcba81a37-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.056712    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-device-dir\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057867    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4ab20462-366d-4e07-a302-25be0aef9dd9-hosts-file\") pod \"node-resolver-bzb4b\" (UID: \"4ab20462-366d-4e07-a302-25be0aef9dd9\") " pod="openshift-dns/node-resolver-bzb4b"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-var-lib-cni-multus\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057992    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-sys-fs\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057981    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-var-lib-openvswitch\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.057949    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33097456-38ba-4136-8f5c-b87cb1707501-host-slash\") pod \"iptables-alerter-hhpxs\" (UID: \"33097456-38ba-4136-8f5c-b87cb1707501\") " pod="openshift-network-operator/iptables-alerter-hhpxs"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058030    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-sys\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.059359 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.057300    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058087    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7cc6e131-076a-4f42-bdc9-874e49b5b938-kubelet-config\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058132    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-registration-dir\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.058139    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret podName:7cc6e131-076a-4f42-bdc9-874e49b5b938 nodeName:}" failed. No retries permitted until 2026-09-14 01:30:43.558117353 +0000 UTC m=+3.038568157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret") pod "global-pull-secret-syncer-dw759" (UID: "7cc6e131-076a-4f42-bdc9-874e49b5b938") : object "kube-system"/"original-pull-secret" not registered
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-var-lib-cni-bin\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058173    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-socket-dir\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058225    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e0a5004-422d-425e-954f-1d2bcba81a37-cnibin\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058243    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-etc-selinux\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058279    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-sysctl-d\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058378    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058443    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/375beafc-69ac-49ff-bf02-5dd15f9d1692-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058476    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-multus-socket-dir-parent\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058528    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vd9l\" (UniqueName: \"kubernetes.io/projected/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-kube-api-access-4vd9l\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-etc-kubernetes\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058634    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-run-systemd\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058668    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ab20462-366d-4e07-a302-25be0aef9dd9-tmp-dir\") pod \"node-resolver-bzb4b\" (UID: \"4ab20462-366d-4e07-a302-25be0aef9dd9\") " pod="openshift-dns/node-resolver-bzb4b"
Sep 14 01:30:43.060286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/86a1cb60-daa6-4531-b896-ea13966bd97c-agent-certs\") pod \"konnectivity-agent-wb7hr\" (UID: \"86a1cb60-daa6-4531-b896-ea13966bd97c\") " pod="kube-system/konnectivity-agent-wb7hr"
Sep 14 01:30:43.061182 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058706    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f00644fe-4d6e-49e3-a485-eb4029ad78ad-serviceca\") pod \"node-ca-rlnvn\" (UID: \"f00644fe-4d6e-49e3-a485-eb4029ad78ad\") " pod="openshift-image-registry/node-ca-rlnvn"
Sep 14 01:30:43.061182 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-multus-cni-dir\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.061182 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058766    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/33097456-38ba-4136-8f5c-b87cb1707501-iptables-alerter-script\") pod \"iptables-alerter-hhpxs\" (UID: \"33097456-38ba-4136-8f5c-b87cb1707501\") " pod="openshift-network-operator/iptables-alerter-hhpxs"
Sep 14 01:30:43.061182 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.058814    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-multus-conf-dir\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.061182 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.059305    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e0a5004-422d-425e-954f-1d2bcba81a37-cni-binary-copy\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:43.061182 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.059445    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ab20462-366d-4e07-a302-25be0aef9dd9-tmp-dir\") pod \"node-resolver-bzb4b\" (UID: \"4ab20462-366d-4e07-a302-25be0aef9dd9\") " pod="openshift-dns/node-resolver-bzb4b"
Sep 14 01:30:43.061182 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.059463    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4e0a5004-422d-425e-954f-1d2bcba81a37-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:43.061182 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.059544    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" event={"ID":"e4be2f7c78788b0bf274c1ec5896ab29","Type":"ContainerStarted","Data":"e1a8619aaad8fa2bf9546391b5dc5e3ad4785aea02503c47a28e4d2bb3af9063"}
Sep 14 01:30:43.066475 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.064938    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 14 01:30:43.069858 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.069824    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 01:30:43.069858 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.069855    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 01:30:43.070030 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.069870    2581 projected.go:196] Error preparing data for projected volume kube-api-access-9j6g8 for pod openshift-network-diagnostics/network-check-target-gg9cz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 01:30:43.070030 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.069956    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8 podName:4ab6aa67-3f53-4c3f-b668-cbeb98b4010e nodeName:}" failed. No retries permitted until 2026-09-14 01:30:43.569934119 +0000 UTC m=+3.050384920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9j6g8" (UniqueName: "kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8") pod "network-check-target-gg9cz" (UID: "4ab6aa67-3f53-4c3f-b668-cbeb98b4010e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 01:30:43.072661 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.072270    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh97l\" (UniqueName: \"kubernetes.io/projected/4ab20462-366d-4e07-a302-25be0aef9dd9-kube-api-access-qh97l\") pod \"node-resolver-bzb4b\" (UID: \"4ab20462-366d-4e07-a302-25be0aef9dd9\") " pod="openshift-dns/node-resolver-bzb4b"
Sep 14 01:30:43.072661 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.072616    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmhdr\" (UniqueName: \"kubernetes.io/projected/375beafc-69ac-49ff-bf02-5dd15f9d1692-kube-api-access-gmhdr\") pod \"aws-ebs-csi-driver-node-kt9r2\" (UID: \"375beafc-69ac-49ff-bf02-5dd15f9d1692\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:43.073315 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.073264    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xkdz\" (UniqueName: \"kubernetes.io/projected/33097456-38ba-4136-8f5c-b87cb1707501-kube-api-access-2xkdz\") pod \"iptables-alerter-hhpxs\" (UID: \"33097456-38ba-4136-8f5c-b87cb1707501\") " pod="openshift-network-operator/iptables-alerter-hhpxs"
Sep 14 01:30:43.073885 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.073840    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrdt4\" (UniqueName: \"kubernetes.io/projected/4e0a5004-422d-425e-954f-1d2bcba81a37-kube-api-access-nrdt4\") pod \"multus-additional-cni-plugins-ktprp\" (UID: \"4e0a5004-422d-425e-954f-1d2bcba81a37\") " pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:43.077914 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.077888    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fsk8\" (UniqueName: \"kubernetes.io/projected/f00644fe-4d6e-49e3-a485-eb4029ad78ad-kube-api-access-4fsk8\") pod \"node-ca-rlnvn\" (UID: \"f00644fe-4d6e-49e3-a485-eb4029ad78ad\") " pod="openshift-image-registry/node-ca-rlnvn"
Sep 14 01:30:43.159681 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.159646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-multus-socket-dir-parent\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.159878 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.159692    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vd9l\" (UniqueName: \"kubernetes.io/projected/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-kube-api-access-4vd9l\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.159878 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.159721    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-etc-kubernetes\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.159878 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.159746    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-run-systemd\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.159878 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.159771    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/86a1cb60-daa6-4531-b896-ea13966bd97c-agent-certs\") pod \"konnectivity-agent-wb7hr\" (UID: \"86a1cb60-daa6-4531-b896-ea13966bd97c\") " pod="kube-system/konnectivity-agent-wb7hr"
Sep 14 01:30:43.159878 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.159796    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-multus-cni-dir\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.159878 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.159824    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-multus-conf-dir\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.159878 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.159850    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dcc0a7d3-0591-4558-899a-0950f6920814-ovn-node-metrics-cert\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.160268 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.159892    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-cnibin\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.160268 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.159923    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-var-lib-kubelet\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.160268 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.159949    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-modprobe-d\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.160268 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.159976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-var-lib-kubelet\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.160268 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.160014    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-tmp\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.160268 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.160043    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-sysconfig\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.160268 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.160067    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-sysctl-conf\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.160268 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.160090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-tuned\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.160268 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.160178    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-multus-socket-dir-parent\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.160268 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.160190    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-multus-conf-dir\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.160268 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.160242    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-run-systemd\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.160268 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.160256    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-etc-kubernetes\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.160872 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.160289    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5b8p\" (UniqueName: \"kubernetes.io/projected/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-kube-api-access-j5b8p\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.160872 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.160377    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-multus-cni-dir\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.160872 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.160422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-var-lib-kubelet\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.160872 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.160537    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-modprobe-d\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.161106 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.160927    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-cnibin\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.161106 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.160985    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-run-netns\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.161106 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161038    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-system-cni-dir\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.161106 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161065    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-run-netns\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.161106 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161092    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-hostroot\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.161384 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-run-ovn-kubernetes\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.161384 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161183    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbpcs\" (UniqueName: \"kubernetes.io/projected/dcc0a7d3-0591-4558-899a-0950f6920814-kube-api-access-sbpcs\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.161384 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-node-log\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.161384 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161235    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-sysconfig\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.161384 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-log-socket\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.161384 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161292    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-kubelet\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.161384 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161336    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-kubelet\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.161724 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161435    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-sysctl-conf\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.161724 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161477    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-hostroot\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.161724 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161514    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-log-socket\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.161724 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161551    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-node-log\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.161724 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161596    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-systemd-units\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.161724 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161598    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-run-netns\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.161724 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-cni-bin\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.161724 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161642    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-run-ovn-kubernetes\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.161724 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161662    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dcc0a7d3-0591-4558-899a-0950f6920814-ovnkube-script-lib\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.161724 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161677    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-run-netns\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.161724 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161709    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-systemd-units\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.162389 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161754    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-cni-bin\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.162389 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161775    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-system-cni-dir\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.162389 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161816    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-var-lib-kubelet\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.162389 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161908    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-run-openvswitch\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.162389 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.161956    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dcc0a7d3-0591-4558-899a-0950f6920814-ovnkube-config\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.162389 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.162031    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-kubernetes\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.162389 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.162065    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-systemd\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.162389 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.162088    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-lib-modules\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.162787 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.162450    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dcc0a7d3-0591-4558-899a-0950f6920814-ovnkube-script-lib\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.162787 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.162595    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dcc0a7d3-0591-4558-899a-0950f6920814-ovnkube-config\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.162787 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.162648    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-run-openvswitch\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.162787 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.162712    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-systemd\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.163045 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.162786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/86a1cb60-daa6-4531-b896-ea13966bd97c-konnectivity-ca\") pod \"konnectivity-agent-wb7hr\" (UID: \"86a1cb60-daa6-4531-b896-ea13966bd97c\") " pod="kube-system/konnectivity-agent-wb7hr"
Sep 14 01:30:43.163045 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.162834    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-cni-binary-copy\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.163045 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.162914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-lib-modules\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.163045 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163022    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-kubernetes\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.163260 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163242    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-run-multus-certs\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.163319 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-cni-netd\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.163377 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163320    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dcc0a7d3-0591-4558-899a-0950f6920814-env-overrides\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.163377 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163368    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:43.163377 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163377    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/86a1cb60-daa6-4531-b896-ea13966bd97c-konnectivity-ca\") pod \"konnectivity-agent-wb7hr\" (UID: \"86a1cb60-daa6-4531-b896-ea13966bd97c\") " pod="kube-system/konnectivity-agent-wb7hr"
Sep 14 01:30:43.163377 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163396    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h5zv\" (UniqueName: \"kubernetes.io/projected/ff2b27ad-c198-489c-9bb8-befeea447a64-kube-api-access-8h5zv\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:43.163609 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.163500    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 01:30:43.163609 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163541    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-cni-netd\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.163609 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163583    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-run-multus-certs\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.163707 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.163653    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs podName:ff2b27ad-c198-489c-9bb8-befeea447a64 nodeName:}" failed. No retries permitted until 2026-09-14 01:30:43.663629501 +0000 UTC m=+3.144080304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs") pod "network-metrics-daemon-fv6nk" (UID: "ff2b27ad-c198-489c-9bb8-befeea447a64") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 01:30:43.163707 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163675    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-etc-openvswitch\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.163780 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163709    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.163780 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163741    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-host\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.163780 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163744    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-etc-openvswitch\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.163780 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-run-ovn\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.163903 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163791    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-tmp\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.163903 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163806    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-multus-daemon-config\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.163903 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163838    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-slash\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.163903 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163868    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-run\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.163903 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-run-k8s-cni-cncf-io\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.164142 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163924    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-var-lib-cni-multus\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.164142 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163950    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-var-lib-openvswitch\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.164142 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163965    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-host\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.164142 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.163975    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-sys\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.164142 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164019    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-slash\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.164142 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164032    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-run-k8s-cni-cncf-io\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.164142 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164076    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-var-lib-cni-bin\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.164142 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164084    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-run\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.164142 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164094    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.164142 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-var-lib-cni-bin\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.164142 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164117    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-host-var-lib-cni-multus\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.164142 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164124    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-run-ovn\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.164142 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-sysctl-d\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.164650 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164161    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dcc0a7d3-0591-4558-899a-0950f6920814-var-lib-openvswitch\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.164650 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164172    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-sys\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.164650 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164333    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-sysctl-d\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.164650 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164360    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-multus-daemon-config\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.164650 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164361    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-cni-binary-copy\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.164650 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-etc-tuned\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.164963 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.164686    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dcc0a7d3-0591-4558-899a-0950f6920814-env-overrides\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.166047 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.166026    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/86a1cb60-daa6-4531-b896-ea13966bd97c-agent-certs\") pod \"konnectivity-agent-wb7hr\" (UID: \"86a1cb60-daa6-4531-b896-ea13966bd97c\") " pod="kube-system/konnectivity-agent-wb7hr"
Sep 14 01:30:43.166818 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.166650    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dcc0a7d3-0591-4558-899a-0950f6920814-ovn-node-metrics-cert\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.170270 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.170128    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vd9l\" (UniqueName: \"kubernetes.io/projected/c10b8a1c-148a-4fdc-86f2-42c2aa6b298b-kube-api-access-4vd9l\") pod \"multus-fcn4g\" (UID: \"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b\") " pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.171248 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.171187    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5b8p\" (UniqueName: \"kubernetes.io/projected/5dff5e3a-d38b-48ed-90a7-bce1b9e3c636-kube-api-access-j5b8p\") pod \"tuned-x5mcp\" (UID: \"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636\") " pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.172240 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.172214    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbpcs\" (UniqueName: \"kubernetes.io/projected/dcc0a7d3-0591-4558-899a-0950f6920814-kube-api-access-sbpcs\") pod \"ovnkube-node-gf7k5\" (UID: \"dcc0a7d3-0591-4558-899a-0950f6920814\") " pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.174387 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.174363    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h5zv\" (UniqueName: \"kubernetes.io/projected/ff2b27ad-c198-489c-9bb8-befeea447a64-kube-api-access-8h5zv\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:43.240158 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.240117    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hhpxs"
Sep 14 01:30:43.250190 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.250079    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bzb4b"
Sep 14 01:30:43.254809 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:43.254778    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33097456_38ba_4136_8f5c_b87cb1707501.slice/crio-c1ec54cb41eaf0fbca8a992ea27fa075dd671e35e7e38438b5a03f822f3e155d WatchSource:0}: Error finding container c1ec54cb41eaf0fbca8a992ea27fa075dd671e35e7e38438b5a03f822f3e155d: Status 404 returned error can't find the container with id c1ec54cb41eaf0fbca8a992ea27fa075dd671e35e7e38438b5a03f822f3e155d
Sep 14 01:30:43.259643 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.259620    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rlnvn"
Sep 14 01:30:43.266029 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.265822    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2"
Sep 14 01:30:43.266127 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:43.266032    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab20462_366d_4e07_a302_25be0aef9dd9.slice/crio-ac082f66d2fe99f58bea008e3cef84998276beff6d53e3f7b433e5e48ed344d8 WatchSource:0}: Error finding container ac082f66d2fe99f58bea008e3cef84998276beff6d53e3f7b433e5e48ed344d8: Status 404 returned error can't find the container with id ac082f66d2fe99f58bea008e3cef84998276beff6d53e3f7b433e5e48ed344d8
Sep 14 01:30:43.271905 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:43.271873    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf00644fe_4d6e_49e3_a485_eb4029ad78ad.slice/crio-16c62c20053dec562789cd1ef0d05e626390731eec44ea95a7c68212127c9242 WatchSource:0}: Error finding container 16c62c20053dec562789cd1ef0d05e626390731eec44ea95a7c68212127c9242: Status 404 returned error can't find the container with id 16c62c20053dec562789cd1ef0d05e626390731eec44ea95a7c68212127c9242
Sep 14 01:30:43.272897 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.272686    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ktprp"
Sep 14 01:30:43.282014 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.281991    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fcn4g"
Sep 14 01:30:43.282900 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:43.282866    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod375beafc_69ac_49ff_bf02_5dd15f9d1692.slice/crio-0023ebd881410ca19567e887d2e340cace50a39501f097383f823e76684aa609 WatchSource:0}: Error finding container 0023ebd881410ca19567e887d2e340cace50a39501f097383f823e76684aa609: Status 404 returned error can't find the container with id 0023ebd881410ca19567e887d2e340cace50a39501f097383f823e76684aa609
Sep 14 01:30:43.290204 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.290174    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:30:43.292223 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:43.292186    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e0a5004_422d_425e_954f_1d2bcba81a37.slice/crio-f9c8479bcdef844f2b931b15f4cc767077fb05baac4c103897bc601dcf0255d8 WatchSource:0}: Error finding container f9c8479bcdef844f2b931b15f4cc767077fb05baac4c103897bc601dcf0255d8: Status 404 returned error can't find the container with id f9c8479bcdef844f2b931b15f4cc767077fb05baac4c103897bc601dcf0255d8
Sep 14 01:30:43.301293 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.301027    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wb7hr"
Sep 14 01:30:43.307130 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:43.307101    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcc0a7d3_0591_4558_899a_0950f6920814.slice/crio-1836ebdf8350544a1195afa6dcd8ede5157ae385315b1668b78d769c8769bbd7 WatchSource:0}: Error finding container 1836ebdf8350544a1195afa6dcd8ede5157ae385315b1668b78d769c8769bbd7: Status 404 returned error can't find the container with id 1836ebdf8350544a1195afa6dcd8ede5157ae385315b1668b78d769c8769bbd7
Sep 14 01:30:43.308375 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.308330    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x5mcp"
Sep 14 01:30:43.313943 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:43.313910    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86a1cb60_daa6_4531_b896_ea13966bd97c.slice/crio-7d96f41d6d3887d741706d916c0dc29f3ce5a1ca3b2eca0e04ee79447387a57d WatchSource:0}: Error finding container 7d96f41d6d3887d741706d916c0dc29f3ce5a1ca3b2eca0e04ee79447387a57d: Status 404 returned error can't find the container with id 7d96f41d6d3887d741706d916c0dc29f3ce5a1ca3b2eca0e04ee79447387a57d
Sep 14 01:30:43.329447 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:30:43.329412    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dff5e3a_d38b_48ed_90a7_bce1b9e3c636.slice/crio-8b0850b3cebe282f1fed5c3ba430e916af7847441fa907dcb2dfdc58d80a9694 WatchSource:0}: Error finding container 8b0850b3cebe282f1fed5c3ba430e916af7847441fa907dcb2dfdc58d80a9694: Status 404 returned error can't find the container with id 8b0850b3cebe282f1fed5c3ba430e916af7847441fa907dcb2dfdc58d80a9694
Sep 14 01:30:43.400140 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.400097    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 01:30:43.568040 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.568006    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:43.568232 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.568149    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 01:30:43.568283 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.568256    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret podName:7cc6e131-076a-4f42-bdc9-874e49b5b938 nodeName:}" failed. No retries permitted until 2026-09-14 01:30:44.568234921 +0000 UTC m=+4.048685728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret") pod "global-pull-secret-syncer-dw759" (UID: "7cc6e131-076a-4f42-bdc9-874e49b5b938") : object "kube-system"/"original-pull-secret" not registered
Sep 14 01:30:43.668788 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.668748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j6g8\" (UniqueName: \"kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8\") pod \"network-check-target-gg9cz\" (UID: \"4ab6aa67-3f53-4c3f-b668-cbeb98b4010e\") " pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:43.668970 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.668809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:43.668970 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.668900    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 01:30:43.668970 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.668914    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 01:30:43.669154 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.668995    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs podName:ff2b27ad-c198-489c-9bb8-befeea447a64 nodeName:}" failed. No retries permitted until 2026-09-14 01:30:44.668977189 +0000 UTC m=+4.149427990 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs") pod "network-metrics-daemon-fv6nk" (UID: "ff2b27ad-c198-489c-9bb8-befeea447a64") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 01:30:43.669154 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.668923    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 01:30:43.669154 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.669045    2581 projected.go:196] Error preparing data for projected volume kube-api-access-9j6g8 for pod openshift-network-diagnostics/network-check-target-gg9cz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 01:30:43.669154 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:43.669115    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8 podName:4ab6aa67-3f53-4c3f-b668-cbeb98b4010e nodeName:}" failed. No retries permitted until 2026-09-14 01:30:44.669093946 +0000 UTC m=+4.149544765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9j6g8" (UniqueName: "kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8") pod "network-check-target-gg9cz" (UID: "4ab6aa67-3f53-4c3f-b668-cbeb98b4010e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 01:30:43.991759 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.991684    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 01:25:43 +0000 UTC" deadline="2028-05-14 13:45:36.799316767 +0000 UTC"
Sep 14 01:30:43.991759 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:43.991758    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14604h14m52.807563356s"
Sep 14 01:30:44.065060 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:44.065016    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wb7hr" event={"ID":"86a1cb60-daa6-4531-b896-ea13966bd97c","Type":"ContainerStarted","Data":"7d96f41d6d3887d741706d916c0dc29f3ce5a1ca3b2eca0e04ee79447387a57d"}
Sep 14 01:30:44.068563 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:44.068523    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktprp" event={"ID":"4e0a5004-422d-425e-954f-1d2bcba81a37","Type":"ContainerStarted","Data":"f9c8479bcdef844f2b931b15f4cc767077fb05baac4c103897bc601dcf0255d8"}
Sep 14 01:30:44.070273 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:44.070243    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2" event={"ID":"375beafc-69ac-49ff-bf02-5dd15f9d1692","Type":"ContainerStarted","Data":"0023ebd881410ca19567e887d2e340cace50a39501f097383f823e76684aa609"}
Sep 14 01:30:44.074465 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:44.074429    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rlnvn" event={"ID":"f00644fe-4d6e-49e3-a485-eb4029ad78ad","Type":"ContainerStarted","Data":"16c62c20053dec562789cd1ef0d05e626390731eec44ea95a7c68212127c9242"}
Sep 14 01:30:44.083843 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:44.083808    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bzb4b" event={"ID":"4ab20462-366d-4e07-a302-25be0aef9dd9","Type":"ContainerStarted","Data":"ac082f66d2fe99f58bea008e3cef84998276beff6d53e3f7b433e5e48ed344d8"}
Sep 14 01:30:44.086476 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:44.086434    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hhpxs" event={"ID":"33097456-38ba-4136-8f5c-b87cb1707501","Type":"ContainerStarted","Data":"c1ec54cb41eaf0fbca8a992ea27fa075dd671e35e7e38438b5a03f822f3e155d"}
Sep 14 01:30:44.089661 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:44.089627    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x5mcp" event={"ID":"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636","Type":"ContainerStarted","Data":"8b0850b3cebe282f1fed5c3ba430e916af7847441fa907dcb2dfdc58d80a9694"}
Sep 14 01:30:44.093001 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:44.092960    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5" event={"ID":"dcc0a7d3-0591-4558-899a-0950f6920814","Type":"ContainerStarted","Data":"1836ebdf8350544a1195afa6dcd8ede5157ae385315b1668b78d769c8769bbd7"}
Sep 14 01:30:44.095517 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:44.095488    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fcn4g" event={"ID":"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b","Type":"ContainerStarted","Data":"f30e66d57c9e7a7ebff7593f4dc6d7863c2012ff9877a22cae2032b08c2ef0ee"}
Sep 14 01:30:44.575621 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:44.575575    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:44.575825 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:44.575751    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 01:30:44.575825 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:44.575820    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret podName:7cc6e131-076a-4f42-bdc9-874e49b5b938 nodeName:}" failed. No retries permitted until 2026-09-14 01:30:46.575800587 +0000 UTC m=+6.056251393 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret") pod "global-pull-secret-syncer-dw759" (UID: "7cc6e131-076a-4f42-bdc9-874e49b5b938") : object "kube-system"/"original-pull-secret" not registered
Sep 14 01:30:44.677104 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:44.677060    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j6g8\" (UniqueName: \"kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8\") pod \"network-check-target-gg9cz\" (UID: \"4ab6aa67-3f53-4c3f-b668-cbeb98b4010e\") " pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:44.677396 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:44.677129    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:44.677396 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:44.677281    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 01:30:44.677396 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:44.677351    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs podName:ff2b27ad-c198-489c-9bb8-befeea447a64 nodeName:}" failed. No retries permitted until 2026-09-14 01:30:46.677331288 +0000 UTC m=+6.157782107 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs") pod "network-metrics-daemon-fv6nk" (UID: "ff2b27ad-c198-489c-9bb8-befeea447a64") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 01:30:44.677570 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:44.677447    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 01:30:44.677570 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:44.677463    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 01:30:44.677570 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:44.677477    2581 projected.go:196] Error preparing data for projected volume kube-api-access-9j6g8 for pod openshift-network-diagnostics/network-check-target-gg9cz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 01:30:44.677570 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:44.677516    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8 podName:4ab6aa67-3f53-4c3f-b668-cbeb98b4010e nodeName:}" failed. No retries permitted until 2026-09-14 01:30:46.677503627 +0000 UTC m=+6.157954444 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9j6g8" (UniqueName: "kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8") pod "network-check-target-gg9cz" (UID: "4ab6aa67-3f53-4c3f-b668-cbeb98b4010e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 01:30:44.992936 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:44.992859    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 01:25:43 +0000 UTC" deadline="2028-03-21 20:53:39.494320669 +0000 UTC"
Sep 14 01:30:44.992936 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:44.992925    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13315h22m54.501414059s"
Sep 14 01:30:45.050782 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:45.049905    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:45.050782 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:45.050053    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:30:45.050782 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:45.050434    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:45.050782 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:45.050523    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:30:45.050782 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:45.050585    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:45.050782 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:45.050669    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:30:46.598563 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:46.597850    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:46.598563 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:46.598050    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 01:30:46.598563 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:46.598136    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret podName:7cc6e131-076a-4f42-bdc9-874e49b5b938 nodeName:}" failed. No retries permitted until 2026-09-14 01:30:50.598117861 +0000 UTC m=+10.078568675 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret") pod "global-pull-secret-syncer-dw759" (UID: "7cc6e131-076a-4f42-bdc9-874e49b5b938") : object "kube-system"/"original-pull-secret" not registered
Sep 14 01:30:46.699390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:46.698544    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j6g8\" (UniqueName: \"kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8\") pod \"network-check-target-gg9cz\" (UID: \"4ab6aa67-3f53-4c3f-b668-cbeb98b4010e\") " pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:46.699390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:46.698599    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:46.699390 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:46.698748    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 01:30:46.699390 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:46.698803    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs podName:ff2b27ad-c198-489c-9bb8-befeea447a64 nodeName:}" failed. No retries permitted until 2026-09-14 01:30:50.69878565 +0000 UTC m=+10.179236449 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs") pod "network-metrics-daemon-fv6nk" (UID: "ff2b27ad-c198-489c-9bb8-befeea447a64") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 01:30:46.699390 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:46.699250    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 01:30:46.699390 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:46.699272    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 01:30:46.699390 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:46.699285    2581 projected.go:196] Error preparing data for projected volume kube-api-access-9j6g8 for pod openshift-network-diagnostics/network-check-target-gg9cz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 01:30:46.699390 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:46.699335    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8 podName:4ab6aa67-3f53-4c3f-b668-cbeb98b4010e nodeName:}" failed. No retries permitted until 2026-09-14 01:30:50.69931774 +0000 UTC m=+10.179768560 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9j6g8" (UniqueName: "kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8") pod "network-check-target-gg9cz" (UID: "4ab6aa67-3f53-4c3f-b668-cbeb98b4010e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 01:30:47.053218 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:47.053132    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:47.053396 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:47.053354    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:30:47.053442 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:47.053144    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:47.053475 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:47.053442    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:47.054940 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:47.053563    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:30:47.054940 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:47.053737    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:30:49.047996 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:49.047964    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:49.047996 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:49.047964    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:49.047996 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:49.047964    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:49.048578 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:49.048128    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:30:49.048578 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:49.048212    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:30:49.048578 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:49.048318    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:30:50.633328 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:50.633285    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:50.634052 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:50.633467    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 01:30:50.634052 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:50.633544    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret podName:7cc6e131-076a-4f42-bdc9-874e49b5b938 nodeName:}" failed. No retries permitted until 2026-09-14 01:30:58.633525147 +0000 UTC m=+18.113975952 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret") pod "global-pull-secret-syncer-dw759" (UID: "7cc6e131-076a-4f42-bdc9-874e49b5b938") : object "kube-system"/"original-pull-secret" not registered
Sep 14 01:30:50.734609 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:50.734567    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:50.734786 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:50.734674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j6g8\" (UniqueName: \"kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8\") pod \"network-check-target-gg9cz\" (UID: \"4ab6aa67-3f53-4c3f-b668-cbeb98b4010e\") " pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:50.734880 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:50.734855    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 01:30:50.734880 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:50.734880    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 01:30:50.735058 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:50.734893    2581 projected.go:196] Error preparing data for projected volume kube-api-access-9j6g8 for pod openshift-network-diagnostics/network-check-target-gg9cz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 01:30:50.735058 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:50.734972    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8 podName:4ab6aa67-3f53-4c3f-b668-cbeb98b4010e nodeName:}" failed. No retries permitted until 2026-09-14 01:30:58.734953299 +0000 UTC m=+18.215404121 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9j6g8" (UniqueName: "kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8") pod "network-check-target-gg9cz" (UID: "4ab6aa67-3f53-4c3f-b668-cbeb98b4010e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 01:30:50.735490 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:50.735467    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 01:30:50.735567 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:50.735540    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs podName:ff2b27ad-c198-489c-9bb8-befeea447a64 nodeName:}" failed. No retries permitted until 2026-09-14 01:30:58.735523309 +0000 UTC m=+18.215974125 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs") pod "network-metrics-daemon-fv6nk" (UID: "ff2b27ad-c198-489c-9bb8-befeea447a64") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 01:30:51.049495 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:51.049226    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:51.049495 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:51.049342    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:30:51.049751 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:51.049583    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:51.049751 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:51.049694    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:30:51.049751 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:51.049716    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:51.049896 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:51.049826    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:30:53.047533 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:53.047499    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:53.048015 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:53.047499    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:53.048015 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:53.047642    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:30:53.048015 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:53.047510    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:53.048015 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:53.047703    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:30:53.048015 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:53.047759    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:30:55.047868 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:55.047826    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:55.048326 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:55.047826    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:55.048326 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:55.047982    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:30:55.048326 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:55.047826    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:55.048326 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:55.048057    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:30:55.048326 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:55.048106    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:30:57.047733 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:57.047687    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:57.047733 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:57.047687    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:57.048347 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:57.047708    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:57.048347 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:57.047827    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:30:57.048347 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:57.047903    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:30:57.048347 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:57.048023    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:30:58.695076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:58.695037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:58.695511 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:58.695209    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 01:30:58.695511 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:58.695287    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret podName:7cc6e131-076a-4f42-bdc9-874e49b5b938 nodeName:}" failed. No retries permitted until 2026-09-14 01:31:14.695266801 +0000 UTC m=+34.175717606 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret") pod "global-pull-secret-syncer-dw759" (UID: "7cc6e131-076a-4f42-bdc9-874e49b5b938") : object "kube-system"/"original-pull-secret" not registered
Sep 14 01:30:58.796396 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:58.796360    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j6g8\" (UniqueName: \"kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8\") pod \"network-check-target-gg9cz\" (UID: \"4ab6aa67-3f53-4c3f-b668-cbeb98b4010e\") " pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:58.796564 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:58.796415    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:58.796564 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:58.796529    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 01:30:58.796564 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:58.796547    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 01:30:58.796564 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:58.796556    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 01:30:58.796775 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:58.796570    2581 projected.go:196] Error preparing data for projected volume kube-api-access-9j6g8 for pod openshift-network-diagnostics/network-check-target-gg9cz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 01:30:58.796775 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:58.796616    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs podName:ff2b27ad-c198-489c-9bb8-befeea447a64 nodeName:}" failed. No retries permitted until 2026-09-14 01:31:14.796597149 +0000 UTC m=+34.277047967 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs") pod "network-metrics-daemon-fv6nk" (UID: "ff2b27ad-c198-489c-9bb8-befeea447a64") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 01:30:58.796775 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:58.796637    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8 podName:4ab6aa67-3f53-4c3f-b668-cbeb98b4010e nodeName:}" failed. No retries permitted until 2026-09-14 01:31:14.796626043 +0000 UTC m=+34.277076848 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9j6g8" (UniqueName: "kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8") pod "network-check-target-gg9cz" (UID: "4ab6aa67-3f53-4c3f-b668-cbeb98b4010e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 01:30:59.047906 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:59.047866    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:30:59.047906 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:59.047888    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:30:59.047906 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:30:59.047866    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:30:59.048187 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:59.048026    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:30:59.048243 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:59.048178    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:30:59.048288 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:30:59.048274    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:31:01.048751 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:01.048714    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:31:01.049199 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:01.048814    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:31:01.049199 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:01.048819    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:31:01.049199 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:01.048851    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:31:01.049199 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:01.048931    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:31:01.049199 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:01.049024    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:31:02.134805 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.134316    2581 generic.go:356] "Generic (PLEG): container finished" podID="e4be2f7c78788b0bf274c1ec5896ab29" containerID="888fb1ef6e06db79d4b9e76632f36b2205497eeec113ed42c70372382418a458" exitCode=0
Sep 14 01:31:02.134805 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.134396    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" event={"ID":"e4be2f7c78788b0bf274c1ec5896ab29","Type":"ContainerDied","Data":"888fb1ef6e06db79d4b9e76632f36b2205497eeec113ed42c70372382418a458"}
Sep 14 01:31:02.136640 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.136595    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x5mcp" event={"ID":"5dff5e3a-d38b-48ed-90a7-bce1b9e3c636","Type":"ContainerStarted","Data":"e0ca8d04f0c20d14a2f891835848fa40ab0daeda7fd30c4fbc28dbd189d4fb66"}
Sep 14 01:31:02.139818 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.139798    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 01:31:02.140176 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.140158    2581 generic.go:356] "Generic (PLEG): container finished" podID="dcc0a7d3-0591-4558-899a-0950f6920814" containerID="7106625b7d29568f9744401d17f824e644f548ae0e880b302e85343f7a2cdfd2" exitCode=1
Sep 14 01:31:02.140248 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.140227    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5" event={"ID":"dcc0a7d3-0591-4558-899a-0950f6920814","Type":"ContainerStarted","Data":"c9f85bed1115ff8db8354c5526b7454ba8a471051c7fb482af81746070a1b55f"}
Sep 14 01:31:02.140371 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.140306    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5" event={"ID":"dcc0a7d3-0591-4558-899a-0950f6920814","Type":"ContainerStarted","Data":"10d17acf66d6d949a4805165522c43bff0b9d531dd7161f58b9dd8df0adcc4c1"}
Sep 14 01:31:02.140371 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.140354    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5" event={"ID":"dcc0a7d3-0591-4558-899a-0950f6920814","Type":"ContainerStarted","Data":"a66e69438ea57885f59b48adb10fd93c74896588c0cd6cce6346d583bb867832"}
Sep 14 01:31:02.140529 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.140394    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5" event={"ID":"dcc0a7d3-0591-4558-899a-0950f6920814","Type":"ContainerStarted","Data":"de7d037f0a0cfc030476720b134cc2c9b93078e9da5d810359275223d2fed480"}
Sep 14 01:31:02.140529 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.140432    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5" event={"ID":"dcc0a7d3-0591-4558-899a-0950f6920814","Type":"ContainerDied","Data":"7106625b7d29568f9744401d17f824e644f548ae0e880b302e85343f7a2cdfd2"}
Sep 14 01:31:02.140529 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.140474    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5" event={"ID":"dcc0a7d3-0591-4558-899a-0950f6920814","Type":"ContainerStarted","Data":"3643acad04abe59ef97178f156b940532951825b5f57f3dc9aff137d4bdc4214"}
Sep 14 01:31:02.141765 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.141729    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fcn4g" event={"ID":"c10b8a1c-148a-4fdc-86f2-42c2aa6b298b","Type":"ContainerStarted","Data":"a6933a55e4b9115c0ee89bca34c37e15608393b27bd2948758372fc0f1cf5c1d"}
Sep 14 01:31:02.143315 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.143294    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal" event={"ID":"9fd5c0f6fc359fb985fe321e9822e84e","Type":"ContainerStarted","Data":"f611c55d7ad48e6c8a4b9a368f5e52d254903d7feb0c9b291c7d9427d297539a"}
Sep 14 01:31:02.146985 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.146959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wb7hr" event={"ID":"86a1cb60-daa6-4531-b896-ea13966bd97c","Type":"ContainerStarted","Data":"4de91893b6e34e24756a2a1318b0861a45e89b048fc3be716a0b6eb1cf32094d"}
Sep 14 01:31:02.148891 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.148867    2581 generic.go:356] "Generic (PLEG): container finished" podID="4e0a5004-422d-425e-954f-1d2bcba81a37" containerID="fe2dc6583eed062c18a276b8c32be0c1ed91091bd537e4d426a4b5e413477bba" exitCode=0
Sep 14 01:31:02.148987 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.148958    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktprp" event={"ID":"4e0a5004-422d-425e-954f-1d2bcba81a37","Type":"ContainerDied","Data":"fe2dc6583eed062c18a276b8c32be0c1ed91091bd537e4d426a4b5e413477bba"}
Sep 14 01:31:02.150539 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.150515    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2" event={"ID":"375beafc-69ac-49ff-bf02-5dd15f9d1692","Type":"ContainerStarted","Data":"01e62b16c612f0eab018c2eb6cc9d8a0c3ece4d139d9476557effe9923621054"}
Sep 14 01:31:02.152167 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.152139    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rlnvn" event={"ID":"f00644fe-4d6e-49e3-a485-eb4029ad78ad","Type":"ContainerStarted","Data":"c6cd8c685cdf78a890992828bfe940bf27fd15d2b6e1490f392caf9a54cf2bb7"}
Sep 14 01:31:02.153553 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.153534    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bzb4b" event={"ID":"4ab20462-366d-4e07-a302-25be0aef9dd9","Type":"ContainerStarted","Data":"da8806051dcb85ad778056b5bcc1dd4856921e6f1b821cd8875736fa7fcbd9dc"}
Sep 14 01:31:02.163581 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.163537    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-11.ec2.internal" podStartSLOduration=20.163520731 podStartE2EDuration="20.163520731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 01:30:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:31:02.162559619 +0000 UTC m=+21.643010440" watchObservedRunningTime="2026-09-14 01:31:02.163520731 +0000 UTC m=+21.643971552"
Sep 14 01:31:02.176786 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.176744    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fcn4g" podStartSLOduration=3.095040913 podStartE2EDuration="21.176725973s" totalImagesPullingTime="18.08168506s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 01:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:31:02.17616823 +0000 UTC m=+21.656619055" watchObservedRunningTime="2026-09-14 01:31:02.176725973 +0000 UTC m=+21.657176795"
Sep 14 01:31:02.193625 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.193582    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-x5mcp" podStartSLOduration=3.186201519 podStartE2EDuration="21.193572371s" totalImagesPullingTime="18.007370852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 01:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:31:02.193564641 +0000 UTC m=+21.674015461" watchObservedRunningTime="2026-09-14 01:31:02.193572371 +0000 UTC m=+21.674023192"
Sep 14 01:31:02.228078 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.228025    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wb7hr" podStartSLOduration=7.159480914 podStartE2EDuration="21.228008355s" totalImagesPullingTime="14.068527441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:31:02.209797066 +0000 UTC m=+21.690247887" watchObservedRunningTime="2026-09-14 01:31:02.228008355 +0000 UTC m=+21.708459175"
Sep 14 01:31:02.228241 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.228212    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bzb4b" podStartSLOduration=3.17161124 podStartE2EDuration="21.228203161s" totalImagesPullingTime="18.056591921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 01:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:31:02.227033668 +0000 UTC m=+21.707484486" watchObservedRunningTime="2026-09-14 01:31:02.228203161 +0000 UTC m=+21.708653982"
Sep 14 01:31:02.949830 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:02.949800    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 14 01:31:03.006811 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:03.006698    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-14T01:31:02.949820825Z","UUID":"4befba6c-1bd0-4358-8e60-e6202672331e","Handler":null,"Name":"","Endpoint":""}
Sep 14 01:31:03.008480 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:03.008445    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 14 01:31:03.008480 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:03.008483    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 14 01:31:03.047763 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:03.047734    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:31:03.047763 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:03.047763    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:31:03.047978 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:03.047837    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:31:03.047978 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:03.047853    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:31:03.047978 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:03.047947    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:31:03.048101 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:03.048024    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:31:03.157577 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:03.157496    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hhpxs" event={"ID":"33097456-38ba-4136-8f5c-b87cb1707501","Type":"ContainerStarted","Data":"49c5d2c20b28795c188c2667d28f1aea043d2e5059669523a16ee4394248673f"}
Sep 14 01:31:03.159661 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:03.159631    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" event={"ID":"e4be2f7c78788b0bf274c1ec5896ab29","Type":"ContainerStarted","Data":"471514229ba1878fbe78470349ac9bbcd44d0fb4b5c9298a192226db2a14b7eb"}
Sep 14 01:31:03.161695 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:03.161671    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2" event={"ID":"375beafc-69ac-49ff-bf02-5dd15f9d1692","Type":"ContainerStarted","Data":"bc28c396b99b449894d943a72e43b6a42daf9a6c691228854c82d3146861d752"}
Sep 14 01:31:03.170540 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:03.170490    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hhpxs" podStartSLOduration=4.088638883 podStartE2EDuration="22.170471706s" totalImagesPullingTime="18.081832823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 01:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:31:03.169930936 +0000 UTC m=+22.650381757" watchObservedRunningTime="2026-09-14 01:31:03.170471706 +0000 UTC m=+22.650922529"
Sep 14 01:31:03.170794 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:03.170768    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rlnvn" podStartSLOduration=4.105338044 podStartE2EDuration="22.170761739s" totalImagesPullingTime="18.065423695s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 01:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:31:02.27637772 +0000 UTC m=+21.756828535" watchObservedRunningTime="2026-09-14 01:31:03.170761739 +0000 UTC m=+22.651212564"
Sep 14 01:31:03.182614 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:03.182575    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-11.ec2.internal" podStartSLOduration=21.182558103 podStartE2EDuration="21.182558103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 01:30:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:31:03.182448647 +0000 UTC m=+22.662899471" watchObservedRunningTime="2026-09-14 01:31:03.182558103 +0000 UTC m=+22.663008925"
Sep 14 01:31:04.167684 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:04.167389    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2" event={"ID":"375beafc-69ac-49ff-bf02-5dd15f9d1692","Type":"ContainerStarted","Data":"8c9aaf7cb03969f821c06d61ee587b2ed1286052c9c51e5aa7532cc87cb5d6ee"}
Sep 14 01:31:04.172161 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:04.172073    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 01:31:04.172657 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:04.172620    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5" event={"ID":"dcc0a7d3-0591-4558-899a-0950f6920814","Type":"ContainerStarted","Data":"8566edb79e91e4f246ded7b291dc44e39080af127bdfe96886ad14d80671c4bd"}
Sep 14 01:31:04.182742 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:04.182698    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kt9r2" podStartSLOduration=7.243724243 podStartE2EDuration="23.182681867s" totalImagesPullingTime="15.938957624s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 01:30:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:31:04.182652474 +0000 UTC m=+23.663103296" watchObservedRunningTime="2026-09-14 01:31:04.182681867 +0000 UTC m=+23.663132679"
Sep 14 01:31:05.047617 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:05.047572    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:31:05.047884 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:05.047572    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:31:05.047884 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:05.047695    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:31:05.047884 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:05.047572    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:31:05.047884 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:05.047858    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:31:05.048101 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:05.047984    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:31:05.466160 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:05.466121    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wb7hr"
Sep 14 01:31:05.467019 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:05.466984    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wb7hr"
Sep 14 01:31:06.177994 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:06.177960    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wb7hr"
Sep 14 01:31:06.178687 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:06.178667    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wb7hr"
Sep 14 01:31:07.048377 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:07.048294    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:31:07.048795 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:07.048296    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:31:07.048795 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:07.048408    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:31:07.048795 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:07.048305    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:31:07.048795 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:07.048495    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:31:07.048945 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:07.048568    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:31:07.183368 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:07.183341    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 01:31:07.183744 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:07.183714    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5" event={"ID":"dcc0a7d3-0591-4558-899a-0950f6920814","Type":"ContainerStarted","Data":"41330f956c0e50ef9c3cce7e16c203447ceb82e5ece5b8f86bc5ed9345d9e559"}
Sep 14 01:31:07.184016 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:07.183986    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:31:07.184136 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:07.184057    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:31:07.184195 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:07.184174    2581 scope.go:117] "RemoveContainer" containerID="7106625b7d29568f9744401d17f824e644f548ae0e880b302e85343f7a2cdfd2"
Sep 14 01:31:07.185869 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:07.185847    2581 generic.go:356] "Generic (PLEG): container finished" podID="4e0a5004-422d-425e-954f-1d2bcba81a37" containerID="5c837cbc21e74737776cd984ad14c0f7bd1c709874b6285ecbed4b6d04674c48" exitCode=0
Sep 14 01:31:07.185979 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:07.185881    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktprp" event={"ID":"4e0a5004-422d-425e-954f-1d2bcba81a37","Type":"ContainerDied","Data":"5c837cbc21e74737776cd984ad14c0f7bd1c709874b6285ecbed4b6d04674c48"}
Sep 14 01:31:07.200372 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:07.200335    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:31:07.200812 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:07.200795    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:31:08.189969 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:08.189870    2581 generic.go:356] "Generic (PLEG): container finished" podID="4e0a5004-422d-425e-954f-1d2bcba81a37" containerID="18d4ac5c3fd4783186492f32d0c7df555158d95679c27e7927e854abc8b3de1e" exitCode=0
Sep 14 01:31:08.190551 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:08.189959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktprp" event={"ID":"4e0a5004-422d-425e-954f-1d2bcba81a37","Type":"ContainerDied","Data":"18d4ac5c3fd4783186492f32d0c7df555158d95679c27e7927e854abc8b3de1e"}
Sep 14 01:31:08.194132 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:08.194114    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 01:31:08.194519 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:08.194493    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5" event={"ID":"dcc0a7d3-0591-4558-899a-0950f6920814","Type":"ContainerStarted","Data":"cd77f0efcf1b238aaa271a6c8a9472ebfbffe0c4e6ffef242e0b4a316bbb7c68"}
Sep 14 01:31:08.194623 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:08.194544    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 14 01:31:08.232990 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:08.232931    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5" podStartSLOduration=9.108884466 podStartE2EDuration="27.232898194s" totalImagesPullingTime="18.124013728s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 01:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:31:08.231697977 +0000 UTC m=+27.712148798" watchObservedRunningTime="2026-09-14 01:31:08.232898194 +0000 UTC m=+27.713349015"
Sep 14 01:31:08.377554 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:08.377512    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gg9cz"]
Sep 14 01:31:08.377734 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:08.377652    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:31:08.377793 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:08.377760    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:31:08.380516 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:08.380489    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fv6nk"]
Sep 14 01:31:08.380670 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:08.380616    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:31:08.380760 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:08.380732    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:31:08.381060 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:08.381035    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dw759"]
Sep 14 01:31:08.381185 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:08.381140    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:31:08.381257 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:08.381219    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:31:09.199558 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:09.199470    2581 generic.go:356] "Generic (PLEG): container finished" podID="4e0a5004-422d-425e-954f-1d2bcba81a37" containerID="7e67a880d2b7acc87dc46cd07d09fda2c14f5bf73a160a363c848bac9264461a" exitCode=0
Sep 14 01:31:09.200054 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:09.199564    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktprp" event={"ID":"4e0a5004-422d-425e-954f-1d2bcba81a37","Type":"ContainerDied","Data":"7e67a880d2b7acc87dc46cd07d09fda2c14f5bf73a160a363c848bac9264461a"}
Sep 14 01:31:09.200054 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:09.199614    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 14 01:31:10.047886 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:10.047844    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:31:10.047886 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:10.047844    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:31:10.048130 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:10.047865    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:31:10.048130 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:10.047991    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:31:10.048239 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:10.048145    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:31:10.048239 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:10.048230    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:31:12.048150 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:12.047865    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:31:12.048702 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:12.047866    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:31:12.048702 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:12.047888    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:31:12.048702 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:12.048305    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:31:12.048702 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:12.048400    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:31:12.048702 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:12.048505    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:31:12.475405 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:12.475368    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:31:12.475649 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:12.475633    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 14 01:31:12.495432 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:12.495377    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5" podUID="dcc0a7d3-0591-4558-899a-0950f6920814" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 14 01:31:12.505104 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:12.505066    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5" podUID="dcc0a7d3-0591-4558-899a-0950f6920814" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 14 01:31:14.047309 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:14.047272    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:31:14.047734 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:14.047283    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:31:14.047734 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:14.047380    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gg9cz" podUID="4ab6aa67-3f53-4c3f-b668-cbeb98b4010e"
Sep 14 01:31:14.047734 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:14.047283    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:31:14.047734 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:14.047447    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dw759" podUID="7cc6e131-076a-4f42-bdc9-874e49b5b938"
Sep 14 01:31:14.047734 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:14.047518    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:31:14.213786 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:14.213751    2581 generic.go:356] "Generic (PLEG): container finished" podID="4e0a5004-422d-425e-954f-1d2bcba81a37" containerID="af274d7206aefcb3af6dfa728700adaab0f3f03c8575b95ebf96349881442639" exitCode=0
Sep 14 01:31:14.214023 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:14.213816    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktprp" event={"ID":"4e0a5004-422d-425e-954f-1d2bcba81a37","Type":"ContainerDied","Data":"af274d7206aefcb3af6dfa728700adaab0f3f03c8575b95ebf96349881442639"}
Sep 14 01:31:14.724722 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:14.724684    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:31:14.724957 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:14.724839    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 01:31:14.724957 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:14.724930    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret podName:7cc6e131-076a-4f42-bdc9-874e49b5b938 nodeName:}" failed. No retries permitted until 2026-09-14 01:31:46.724894101 +0000 UTC m=+66.205344917 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret") pod "global-pull-secret-syncer-dw759" (UID: "7cc6e131-076a-4f42-bdc9-874e49b5b938") : object "kube-system"/"original-pull-secret" not registered
Sep 14 01:31:14.825189 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:14.825153    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:31:14.825405 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:14.825223    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j6g8\" (UniqueName: \"kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8\") pod \"network-check-target-gg9cz\" (UID: \"4ab6aa67-3f53-4c3f-b668-cbeb98b4010e\") " pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:31:14.825405 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:14.825302    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 01:31:14.825405 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:14.825338    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 01:31:14.825405 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:14.825362    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 01:31:14.825405 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:14.825381    2581 projected.go:196] Error preparing data for projected volume kube-api-access-9j6g8 for pod openshift-network-diagnostics/network-check-target-gg9cz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 01:31:14.825604 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:14.825365    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs podName:ff2b27ad-c198-489c-9bb8-befeea447a64 nodeName:}" failed. No retries permitted until 2026-09-14 01:31:46.825349921 +0000 UTC m=+66.305800720 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs") pod "network-metrics-daemon-fv6nk" (UID: "ff2b27ad-c198-489c-9bb8-befeea447a64") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 01:31:14.825604 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:14.825435    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8 podName:4ab6aa67-3f53-4c3f-b668-cbeb98b4010e nodeName:}" failed. No retries permitted until 2026-09-14 01:31:46.825421345 +0000 UTC m=+66.305872158 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9j6g8" (UniqueName: "kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8") pod "network-check-target-gg9cz" (UID: "4ab6aa67-3f53-4c3f-b668-cbeb98b4010e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 01:31:15.219227 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.219194    2581 generic.go:356] "Generic (PLEG): container finished" podID="4e0a5004-422d-425e-954f-1d2bcba81a37" containerID="0e98a8ca5ee2725d2a0ad8599f2580366590893bc565fcc33a011821b8a55ff5" exitCode=0
Sep 14 01:31:15.219754 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.219247    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktprp" event={"ID":"4e0a5004-422d-425e-954f-1d2bcba81a37","Type":"ContainerDied","Data":"0e98a8ca5ee2725d2a0ad8599f2580366590893bc565fcc33a011821b8a55ff5"}
Sep 14 01:31:15.310669 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.310628    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-11.ec2.internal" event="NodeReady"
Sep 14 01:31:15.310815 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.310796    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 14 01:31:15.341058 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.341027    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56cb6897db-dvtww"]
Sep 14 01:31:15.357155 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.357128    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"]
Sep 14 01:31:15.357310 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.357287    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.359365 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.359343    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 14 01:31:15.359746 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.359707    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 14 01:31:15.360104 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.359934    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 14 01:31:15.360104 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.359995    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vh46f\""
Sep 14 01:31:15.365970 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.365951    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 14 01:31:15.375472 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.375447    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5fs9z"]
Sep 14 01:31:15.375620 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.375553    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:31:15.377550 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.377527    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 14 01:31:15.377550 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.377541    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 14 01:31:15.377705 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.377599    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jt2tm\""
Sep 14 01:31:15.399619 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.399595    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jxz4h"]
Sep 14 01:31:15.399739 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.399726    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5fs9z"
Sep 14 01:31:15.401471 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.401444    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 14 01:31:15.401691 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.401664    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 14 01:31:15.401792 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.401688    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tbbp2\""
Sep 14 01:31:15.401792 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.401719    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 14 01:31:15.423664 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.423638    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"]
Sep 14 01:31:15.423761 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.423672    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56cb6897db-dvtww"]
Sep 14 01:31:15.423761 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.423693    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5fs9z"]
Sep 14 01:31:15.423761 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.423712    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jxz4h"]
Sep 14 01:31:15.423858 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.423764    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:15.425478 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.425453    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 14 01:31:15.425478 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.425476    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 14 01:31:15.425636 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.425497    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jfmw6\""
Sep 14 01:31:15.491116 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.491035    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wb7hr_86a1cb60-daa6-4531-b896-ea13966bd97c/konnectivity-agent/0.log" limit=1024
Sep 14 01:31:15.530061 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530029    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:15.530177 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530065    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74829c0c-ae15-4b20-b653-1e23c44c899b-cert\") pod \"ingress-canary-5fs9z\" (UID: \"74829c0c-ae15-4b20-b653-1e23c44c899b\") " pod="openshift-ingress-canary/ingress-canary-5fs9z"
Sep 14 01:31:15.530177 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530089    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsdpc\" (UniqueName: \"kubernetes.io/projected/248acebf-b6bf-407e-ad54-0976487a1b2c-kube-api-access-wsdpc\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:15.530177 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530112    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnmzs\" (UniqueName: \"kubernetes.io/projected/74829c0c-ae15-4b20-b653-1e23c44c899b-kube-api-access-wnmzs\") pod \"ingress-canary-5fs9z\" (UID: \"74829c0c-ae15-4b20-b653-1e23c44c899b\") " pod="openshift-ingress-canary/ingress-canary-5fs9z"
Sep 14 01:31:15.530177 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/248acebf-b6bf-407e-ad54-0976487a1b2c-config-volume\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:15.530177 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530175    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.530371 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-certificates\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.530371 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530220    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/526bc8e9-328b-4c29-912e-1e092a8b97fa-installation-pull-secrets\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.530371 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530267    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7qnf\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-kube-api-access-h7qnf\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.530371 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530308    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9935e3f9-36eb-4f10-9022-9a2699cf0866-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-xhr5r\" (UID: \"9935e3f9-36eb-4f10-9022-9a2699cf0866\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:31:15.530490 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530381    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-bound-sa-token\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.530490 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530401    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/526bc8e9-328b-4c29-912e-1e092a8b97fa-ca-trust-extracted\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.530490 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530444    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/248acebf-b6bf-407e-ad54-0976487a1b2c-tmp-dir\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:15.530490 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530462    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/526bc8e9-328b-4c29-912e-1e092a8b97fa-image-registry-private-configuration\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.530490 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530485    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xhr5r\" (UID: \"9935e3f9-36eb-4f10-9022-9a2699cf0866\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:31:15.530636 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.530508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/526bc8e9-328b-4c29-912e-1e092a8b97fa-trusted-ca\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.570070 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.570034    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wb7hr_86a1cb60-daa6-4531-b896-ea13966bd97c/konnectivity-agent/0.log" limit=1024
Sep 14 01:31:15.631033 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.630993    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/526bc8e9-328b-4c29-912e-1e092a8b97fa-installation-pull-secrets\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.631033 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631036    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7qnf\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-kube-api-access-h7qnf\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.631238 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631052    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9935e3f9-36eb-4f10-9022-9a2699cf0866-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-xhr5r\" (UID: \"9935e3f9-36eb-4f10-9022-9a2699cf0866\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:31:15.631238 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631082    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-bound-sa-token\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.631315 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631286    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/526bc8e9-328b-4c29-912e-1e092a8b97fa-ca-trust-extracted\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.631363 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/248acebf-b6bf-407e-ad54-0976487a1b2c-tmp-dir\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:15.631405 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/526bc8e9-328b-4c29-912e-1e092a8b97fa-image-registry-private-configuration\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.631447 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xhr5r\" (UID: \"9935e3f9-36eb-4f10-9022-9a2699cf0866\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:31:15.631447 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631438    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/526bc8e9-328b-4c29-912e-1e092a8b97fa-trusted-ca\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.631541 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:15.631541 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74829c0c-ae15-4b20-b653-1e23c44c899b-cert\") pod \"ingress-canary-5fs9z\" (UID: \"74829c0c-ae15-4b20-b653-1e23c44c899b\") " pod="openshift-ingress-canary/ingress-canary-5fs9z"
Sep 14 01:31:15.631541 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631511    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsdpc\" (UniqueName: \"kubernetes.io/projected/248acebf-b6bf-407e-ad54-0976487a1b2c-kube-api-access-wsdpc\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:15.631541 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnmzs\" (UniqueName: \"kubernetes.io/projected/74829c0c-ae15-4b20-b653-1e23c44c899b-kube-api-access-wnmzs\") pod \"ingress-canary-5fs9z\" (UID: \"74829c0c-ae15-4b20-b653-1e23c44c899b\") " pod="openshift-ingress-canary/ingress-canary-5fs9z"
Sep 14 01:31:15.631730 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631564    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/248acebf-b6bf-407e-ad54-0976487a1b2c-config-volume\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:15.631730 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:15.631595    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 14 01:31:15.631730 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.631730 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631628    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-certificates\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.631730 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631667    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/248acebf-b6bf-407e-ad54-0976487a1b2c-tmp-dir\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:15.631730 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:15.631678    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert podName:9935e3f9-36eb-4f10-9022-9a2699cf0866 nodeName:}" failed. No retries permitted until 2026-09-14 01:31:16.131642722 +0000 UTC m=+35.612093536 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-xhr5r" (UID: "9935e3f9-36eb-4f10-9022-9a2699cf0866") : secret "networking-console-plugin-cert" not found
Sep 14 01:31:15.631730 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631697    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/526bc8e9-328b-4c29-912e-1e092a8b97fa-ca-trust-extracted\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.632101 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:15.631896    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 01:31:15.632101 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.631905    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9935e3f9-36eb-4f10-9022-9a2699cf0866-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-xhr5r\" (UID: \"9935e3f9-36eb-4f10-9022-9a2699cf0866\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:31:15.632101 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:15.631965    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls podName:248acebf-b6bf-407e-ad54-0976487a1b2c nodeName:}" failed. No retries permitted until 2026-09-14 01:31:16.131948556 +0000 UTC m=+35.612399365 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls") pod "dns-default-jxz4h" (UID: "248acebf-b6bf-407e-ad54-0976487a1b2c") : secret "dns-default-metrics-tls" not found
Sep 14 01:31:15.632101 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:15.632039    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 01:31:15.632101 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:15.632050    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56cb6897db-dvtww: secret "image-registry-tls" not found
Sep 14 01:31:15.632101 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:15.632083    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls podName:526bc8e9-328b-4c29-912e-1e092a8b97fa nodeName:}" failed. No retries permitted until 2026-09-14 01:31:16.132072731 +0000 UTC m=+35.612523541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls") pod "image-registry-56cb6897db-dvtww" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa") : secret "image-registry-tls" not found
Sep 14 01:31:15.632385 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.632268    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-certificates\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.632507 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.632482    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/248acebf-b6bf-407e-ad54-0976487a1b2c-config-volume\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:15.632720 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.632700    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/526bc8e9-328b-4c29-912e-1e092a8b97fa-trusted-ca\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.635847 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.635740    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/526bc8e9-328b-4c29-912e-1e092a8b97fa-installation-pull-secrets\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.635982 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.635780    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/526bc8e9-328b-4c29-912e-1e092a8b97fa-image-registry-private-configuration\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.635982 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.635810    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74829c0c-ae15-4b20-b653-1e23c44c899b-cert\") pod \"ingress-canary-5fs9z\" (UID: \"74829c0c-ae15-4b20-b653-1e23c44c899b\") " pod="openshift-ingress-canary/ingress-canary-5fs9z"
Sep 14 01:31:15.641410 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.641384    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsdpc\" (UniqueName: \"kubernetes.io/projected/248acebf-b6bf-407e-ad54-0976487a1b2c-kube-api-access-wsdpc\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:15.641687 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.641667    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnmzs\" (UniqueName: \"kubernetes.io/projected/74829c0c-ae15-4b20-b653-1e23c44c899b-kube-api-access-wnmzs\") pod \"ingress-canary-5fs9z\" (UID: \"74829c0c-ae15-4b20-b653-1e23c44c899b\") " pod="openshift-ingress-canary/ingress-canary-5fs9z"
Sep 14 01:31:15.641804 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.641771    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7qnf\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-kube-api-access-h7qnf\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.642066 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.642049    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-bound-sa-token\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:15.708224 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:15.708185    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5fs9z"
Sep 14 01:31:16.018403 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.018368    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5fs9z"]
Sep 14 01:31:16.022786 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:31:16.022756    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74829c0c_ae15_4b20_b653_1e23c44c899b.slice/crio-b19a22fa2be56d3c4f93fbcba9058ab5ceef1249bb1492615310892d48b69433 WatchSource:0}: Error finding container b19a22fa2be56d3c4f93fbcba9058ab5ceef1249bb1492615310892d48b69433: Status 404 returned error can't find the container with id b19a22fa2be56d3c4f93fbcba9058ab5ceef1249bb1492615310892d48b69433
Sep 14 01:31:16.048395 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.048323    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:31:16.048516 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.048333    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:31:16.048516 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.048432    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:31:16.050410 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.050387    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 01:31:16.050527 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.050416    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 01:31:16.050527 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.050496    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gv4zq\""
Sep 14 01:31:16.050644 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.050532    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tck5c\""
Sep 14 01:31:16.050644 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.050602    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 01:31:16.050808 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.050791    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 01:31:16.136558 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.136521    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xhr5r\" (UID: \"9935e3f9-36eb-4f10-9022-9a2699cf0866\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:31:16.136770 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.136566    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:16.136770 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.136608    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:16.136770 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:16.136683    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 14 01:31:16.136770 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:16.136698    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 01:31:16.136770 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:16.136709    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56cb6897db-dvtww: secret "image-registry-tls" not found
Sep 14 01:31:16.136770 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:16.136710    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 01:31:16.136770 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:16.136754    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert podName:9935e3f9-36eb-4f10-9022-9a2699cf0866 nodeName:}" failed. No retries permitted until 2026-09-14 01:31:17.136734659 +0000 UTC m=+36.617185458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-xhr5r" (UID: "9935e3f9-36eb-4f10-9022-9a2699cf0866") : secret "networking-console-plugin-cert" not found
Sep 14 01:31:16.136770 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:16.136774    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls podName:526bc8e9-328b-4c29-912e-1e092a8b97fa nodeName:}" failed. No retries permitted until 2026-09-14 01:31:17.136763537 +0000 UTC m=+36.617214336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls") pod "image-registry-56cb6897db-dvtww" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa") : secret "image-registry-tls" not found
Sep 14 01:31:16.137100 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:16.136829    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls podName:248acebf-b6bf-407e-ad54-0976487a1b2c nodeName:}" failed. No retries permitted until 2026-09-14 01:31:17.136819242 +0000 UTC m=+36.617270042 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls") pod "dns-default-jxz4h" (UID: "248acebf-b6bf-407e-ad54-0976487a1b2c") : secret "dns-default-metrics-tls" not found
Sep 14 01:31:16.224816 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.224779    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktprp" event={"ID":"4e0a5004-422d-425e-954f-1d2bcba81a37","Type":"ContainerStarted","Data":"28fb7e324d95b5d085cb498c95a2d6b2065a16ff192630b45bf59ffcca4b626f"}
Sep 14 01:31:16.226071 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.226042    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5fs9z" event={"ID":"74829c0c-ae15-4b20-b653-1e23c44c899b","Type":"ContainerStarted","Data":"b19a22fa2be56d3c4f93fbcba9058ab5ceef1249bb1492615310892d48b69433"}
Sep 14 01:31:16.249236 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:16.249189    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ktprp" podStartSLOduration=7.2096939110000005 podStartE2EDuration="35.249173017s" totalImagesPullingTime="28.039479106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 01:30:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:31:16.247528174 +0000 UTC m=+35.727979244" watchObservedRunningTime="2026-09-14 01:31:16.249173017 +0000 UTC m=+35.729623837"
Sep 14 01:31:17.146066 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:17.146034    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:17.146258 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:17.146102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:17.146258 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:17.146177    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xhr5r\" (UID: \"9935e3f9-36eb-4f10-9022-9a2699cf0866\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:31:17.146258 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:17.146211    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 01:31:17.146258 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:17.146247    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 01:31:17.146468 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:17.146265    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56cb6897db-dvtww: secret "image-registry-tls" not found
Sep 14 01:31:17.146468 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:17.146284    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls podName:248acebf-b6bf-407e-ad54-0976487a1b2c nodeName:}" failed. No retries permitted until 2026-09-14 01:31:19.146263033 +0000 UTC m=+38.626713838 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls") pod "dns-default-jxz4h" (UID: "248acebf-b6bf-407e-ad54-0976487a1b2c") : secret "dns-default-metrics-tls" not found
Sep 14 01:31:17.146468 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:17.146287    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 14 01:31:17.146468 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:17.146328    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls podName:526bc8e9-328b-4c29-912e-1e092a8b97fa nodeName:}" failed. No retries permitted until 2026-09-14 01:31:19.146311396 +0000 UTC m=+38.626762214 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls") pod "image-registry-56cb6897db-dvtww" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa") : secret "image-registry-tls" not found
Sep 14 01:31:17.146468 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:17.146345    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert podName:9935e3f9-36eb-4f10-9022-9a2699cf0866 nodeName:}" failed. No retries permitted until 2026-09-14 01:31:19.146335556 +0000 UTC m=+38.626786361 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-xhr5r" (UID: "9935e3f9-36eb-4f10-9022-9a2699cf0866") : secret "networking-console-plugin-cert" not found
Sep 14 01:31:19.164203 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:19.164154    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xhr5r\" (UID: \"9935e3f9-36eb-4f10-9022-9a2699cf0866\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:31:19.164203 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:19.164211    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:19.164968 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:19.164247    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:19.164968 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:19.164315    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 14 01:31:19.164968 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:19.164381    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert podName:9935e3f9-36eb-4f10-9022-9a2699cf0866 nodeName:}" failed. No retries permitted until 2026-09-14 01:31:23.164364181 +0000 UTC m=+42.644814994 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-xhr5r" (UID: "9935e3f9-36eb-4f10-9022-9a2699cf0866") : secret "networking-console-plugin-cert" not found
Sep 14 01:31:19.164968 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:19.164385    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 01:31:19.164968 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:19.164444    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls podName:248acebf-b6bf-407e-ad54-0976487a1b2c nodeName:}" failed. No retries permitted until 2026-09-14 01:31:23.164427951 +0000 UTC m=+42.644878750 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls") pod "dns-default-jxz4h" (UID: "248acebf-b6bf-407e-ad54-0976487a1b2c") : secret "dns-default-metrics-tls" not found
Sep 14 01:31:19.164968 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:19.164447    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 01:31:19.164968 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:19.164463    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56cb6897db-dvtww: secret "image-registry-tls" not found
Sep 14 01:31:19.164968 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:19.164506    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls podName:526bc8e9-328b-4c29-912e-1e092a8b97fa nodeName:}" failed. No retries permitted until 2026-09-14 01:31:23.164494139 +0000 UTC m=+42.644944937 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls") pod "image-registry-56cb6897db-dvtww" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa") : secret "image-registry-tls" not found
Sep 14 01:31:19.235560 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:19.235529    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5fs9z" event={"ID":"74829c0c-ae15-4b20-b653-1e23c44c899b","Type":"ContainerStarted","Data":"17e0937c248aa9e820bc146808bc551f2666f113d9115b043a163fe68c4206a2"}
Sep 14 01:31:19.249082 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:19.249038    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5fs9z" podStartSLOduration=1.747643129 podStartE2EDuration="4.249022487s" totalImagesPullingTime="2.501379358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:31:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:31:19.24805746 +0000 UTC m=+38.728508285" watchObservedRunningTime="2026-09-14 01:31:19.249022487 +0000 UTC m=+38.729473326"
Sep 14 01:31:23.196902 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:23.196644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:23.196902 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:23.196892    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xhr5r\" (UID: \"9935e3f9-36eb-4f10-9022-9a2699cf0866\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:31:23.197403 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:23.196817    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 01:31:23.197403 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:23.196940    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:23.197403 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:23.196943    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56cb6897db-dvtww: secret "image-registry-tls" not found
Sep 14 01:31:23.197403 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:23.197102    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls podName:526bc8e9-328b-4c29-912e-1e092a8b97fa nodeName:}" failed. No retries permitted until 2026-09-14 01:31:31.19708818 +0000 UTC m=+50.677538991 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls") pod "image-registry-56cb6897db-dvtww" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa") : secret "image-registry-tls" not found
Sep 14 01:31:23.197403 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:23.196989    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 14 01:31:23.197403 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:23.197134    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 01:31:23.197403 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:23.197247    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls podName:248acebf-b6bf-407e-ad54-0976487a1b2c nodeName:}" failed. No retries permitted until 2026-09-14 01:31:31.197226074 +0000 UTC m=+50.677676887 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls") pod "dns-default-jxz4h" (UID: "248acebf-b6bf-407e-ad54-0976487a1b2c") : secret "dns-default-metrics-tls" not found
Sep 14 01:31:23.197403 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:23.197269    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert podName:9935e3f9-36eb-4f10-9022-9a2699cf0866 nodeName:}" failed. No retries permitted until 2026-09-14 01:31:31.197259328 +0000 UTC m=+50.677710142 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-xhr5r" (UID: "9935e3f9-36eb-4f10-9022-9a2699cf0866") : secret "networking-console-plugin-cert" not found
Sep 14 01:31:31.251574 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:31.251536    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xhr5r\" (UID: \"9935e3f9-36eb-4f10-9022-9a2699cf0866\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:31:31.251574 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:31.251581    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:31.252190 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:31.251628    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:31.252190 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:31.251719    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 14 01:31:31.252190 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:31.251772    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 01:31:31.252190 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:31.251793    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56cb6897db-dvtww: secret "image-registry-tls" not found
Sep 14 01:31:31.252190 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:31.251804    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert podName:9935e3f9-36eb-4f10-9022-9a2699cf0866 nodeName:}" failed. No retries permitted until 2026-09-14 01:31:47.251786616 +0000 UTC m=+66.732237430 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-xhr5r" (UID: "9935e3f9-36eb-4f10-9022-9a2699cf0866") : secret "networking-console-plugin-cert" not found
Sep 14 01:31:31.252190 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:31.251847    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls podName:526bc8e9-328b-4c29-912e-1e092a8b97fa nodeName:}" failed. No retries permitted until 2026-09-14 01:31:47.251834076 +0000 UTC m=+66.732284890 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls") pod "image-registry-56cb6897db-dvtww" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa") : secret "image-registry-tls" not found
Sep 14 01:31:31.252190 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:31.251774    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 01:31:31.252190 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:31.251965    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls podName:248acebf-b6bf-407e-ad54-0976487a1b2c nodeName:}" failed. No retries permitted until 2026-09-14 01:31:47.251947775 +0000 UTC m=+66.732398575 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls") pod "dns-default-jxz4h" (UID: "248acebf-b6bf-407e-ad54-0976487a1b2c") : secret "dns-default-metrics-tls" not found
Sep 14 01:31:42.506423 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:42.506395    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gf7k5"
Sep 14 01:31:46.767037 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:46.766988    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:31:46.769223 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:46.769204    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 01:31:46.780520 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:46.780495    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cc6e131-076a-4f42-bdc9-874e49b5b938-original-pull-secret\") pod \"global-pull-secret-syncer-dw759\" (UID: \"7cc6e131-076a-4f42-bdc9-874e49b5b938\") " pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:31:46.867893 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:46.867857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j6g8\" (UniqueName: \"kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8\") pod \"network-check-target-gg9cz\" (UID: \"4ab6aa67-3f53-4c3f-b668-cbeb98b4010e\") " pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:31:46.868088 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:46.867904    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:31:46.870452 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:46.870429    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 01:31:46.870583 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:46.870564    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 01:31:46.878193 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:46.878171    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 14 01:31:46.878247 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:46.878238    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs podName:ff2b27ad-c198-489c-9bb8-befeea447a64 nodeName:}" failed. No retries permitted until 2026-09-14 01:32:50.878218566 +0000 UTC m=+130.358669366 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs") pod "network-metrics-daemon-fv6nk" (UID: "ff2b27ad-c198-489c-9bb8-befeea447a64") : secret "metrics-daemon-secret" not found
Sep 14 01:31:46.879782 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:46.879766    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 01:31:46.891405 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:46.891374    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j6g8\" (UniqueName: \"kubernetes.io/projected/4ab6aa67-3f53-4c3f-b668-cbeb98b4010e-kube-api-access-9j6g8\") pod \"network-check-target-gg9cz\" (UID: \"4ab6aa67-3f53-4c3f-b668-cbeb98b4010e\") " pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:31:46.960518 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:46.960486    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tck5c\""
Sep 14 01:31:46.963964 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:46.963943    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dw759"
Sep 14 01:31:46.969417 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:46.969400    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:31:47.236513 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:47.236477    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gg9cz"]
Sep 14 01:31:47.239405 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:47.239381    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dw759"]
Sep 14 01:31:47.241105 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:31:47.241078    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab6aa67_3f53_4c3f_b668_cbeb98b4010e.slice/crio-9b333707a9ff9da18c3da28c59290138525b649231096a85d60f0b995fa50853 WatchSource:0}: Error finding container 9b333707a9ff9da18c3da28c59290138525b649231096a85d60f0b995fa50853: Status 404 returned error can't find the container with id 9b333707a9ff9da18c3da28c59290138525b649231096a85d60f0b995fa50853
Sep 14 01:31:47.242020 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:31:47.242001    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc6e131_076a_4f42_bdc9_874e49b5b938.slice/crio-acee30878a198e2df72665870b2b0bafc2253817ba376706c1fe0fc17bbf4ee4 WatchSource:0}: Error finding container acee30878a198e2df72665870b2b0bafc2253817ba376706c1fe0fc17bbf4ee4: Status 404 returned error can't find the container with id acee30878a198e2df72665870b2b0bafc2253817ba376706c1fe0fc17bbf4ee4
Sep 14 01:31:47.271450 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:47.271363    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xhr5r\" (UID: \"9935e3f9-36eb-4f10-9022-9a2699cf0866\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:31:47.271450 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:47.271404    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:31:47.271450 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:47.271434    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:31:47.271650 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:47.271506    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 14 01:31:47.271650 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:47.271536    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 01:31:47.271650 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:47.271548    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56cb6897db-dvtww: secret "image-registry-tls" not found
Sep 14 01:31:47.271650 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:47.271554    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 01:31:47.271650 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:47.271572    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert podName:9935e3f9-36eb-4f10-9022-9a2699cf0866 nodeName:}" failed. No retries permitted until 2026-09-14 01:32:19.271556224 +0000 UTC m=+98.752007037 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-xhr5r" (UID: "9935e3f9-36eb-4f10-9022-9a2699cf0866") : secret "networking-console-plugin-cert" not found
Sep 14 01:31:47.271650 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:47.271586    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls podName:526bc8e9-328b-4c29-912e-1e092a8b97fa nodeName:}" failed. No retries permitted until 2026-09-14 01:32:19.271579837 +0000 UTC m=+98.752030636 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls") pod "image-registry-56cb6897db-dvtww" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa") : secret "image-registry-tls" not found
Sep 14 01:31:47.271650 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:31:47.271598    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls podName:248acebf-b6bf-407e-ad54-0976487a1b2c nodeName:}" failed. No retries permitted until 2026-09-14 01:32:19.27159255 +0000 UTC m=+98.752043348 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls") pod "dns-default-jxz4h" (UID: "248acebf-b6bf-407e-ad54-0976487a1b2c") : secret "dns-default-metrics-tls" not found
Sep 14 01:31:47.302304 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:47.302269    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gg9cz" event={"ID":"4ab6aa67-3f53-4c3f-b668-cbeb98b4010e","Type":"ContainerStarted","Data":"9b333707a9ff9da18c3da28c59290138525b649231096a85d60f0b995fa50853"}
Sep 14 01:31:47.303439 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:47.303410    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dw759" event={"ID":"7cc6e131-076a-4f42-bdc9-874e49b5b938","Type":"ContainerStarted","Data":"acee30878a198e2df72665870b2b0bafc2253817ba376706c1fe0fc17bbf4ee4"}
Sep 14 01:31:52.320364 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:52.320322    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gg9cz" event={"ID":"4ab6aa67-3f53-4c3f-b668-cbeb98b4010e","Type":"ContainerStarted","Data":"8a363d3c1120e101c0cf057ca25ba7ee866667eab3d8c7316b3838a443390094"}
Sep 14 01:31:52.320955 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:52.320454    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:31:52.321883 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:52.321864    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dw759" event={"ID":"7cc6e131-076a-4f42-bdc9-874e49b5b938","Type":"ContainerStarted","Data":"8c1b8759e25309b2698a9ab46d190acb0a26098414dc14b0571a05b674968385"}
Sep 14 01:31:52.333299 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:52.333246    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gg9cz" podStartSLOduration=66.934029068 podStartE2EDuration="1m11.333233615s" totalImagesPullingTime="4.399204547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:31:52.332961735 +0000 UTC m=+71.813412560" watchObservedRunningTime="2026-09-14 01:31:52.333233615 +0000 UTC m=+71.813684429"
Sep 14 01:31:52.344597 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:31:52.344545    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dw759" podStartSLOduration=66.941494702 podStartE2EDuration="1m11.344528748s" totalImagesPullingTime="4.403034046s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 01:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:31:52.344273834 +0000 UTC m=+71.824724657" watchObservedRunningTime="2026-09-14 01:31:52.344528748 +0000 UTC m=+71.824979569"
Sep 14 01:32:19.304557 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:32:19.304516    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:32:19.305084 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:32:19.304578    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:32:19.305084 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:32:19.304640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xhr5r\" (UID: \"9935e3f9-36eb-4f10-9022-9a2699cf0866\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:32:19.305084 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:32:19.304669    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 01:32:19.305084 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:32:19.304728    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 01:32:19.305084 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:32:19.304744    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56cb6897db-dvtww: secret "image-registry-tls" not found
Sep 14 01:32:19.305084 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:32:19.304746    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 14 01:32:19.305084 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:32:19.304734    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls podName:248acebf-b6bf-407e-ad54-0976487a1b2c nodeName:}" failed. No retries permitted until 2026-09-14 01:33:23.304716131 +0000 UTC m=+162.785166929 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls") pod "dns-default-jxz4h" (UID: "248acebf-b6bf-407e-ad54-0976487a1b2c") : secret "dns-default-metrics-tls" not found
Sep 14 01:32:19.305084 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:32:19.304807    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls podName:526bc8e9-328b-4c29-912e-1e092a8b97fa nodeName:}" failed. No retries permitted until 2026-09-14 01:33:23.304794649 +0000 UTC m=+162.785245448 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls") pod "image-registry-56cb6897db-dvtww" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa") : secret "image-registry-tls" not found
Sep 14 01:32:19.305084 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:32:19.304823    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert podName:9935e3f9-36eb-4f10-9022-9a2699cf0866 nodeName:}" failed. No retries permitted until 2026-09-14 01:33:23.304816054 +0000 UTC m=+162.785266852 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-xhr5r" (UID: "9935e3f9-36eb-4f10-9022-9a2699cf0866") : secret "networking-console-plugin-cert" not found
Sep 14 01:32:23.326551 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:32:23.326522    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gg9cz"
Sep 14 01:32:50.930728 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:32:50.930668    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:32:50.931262 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:32:50.930815    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 14 01:32:50.931262 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:32:50.930880    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs podName:ff2b27ad-c198-489c-9bb8-befeea447a64 nodeName:}" failed. No retries permitted until 2026-09-14 01:34:52.930865173 +0000 UTC m=+252.411315972 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs") pod "network-metrics-daemon-fv6nk" (UID: "ff2b27ad-c198-489c-9bb8-befeea447a64") : secret "metrics-daemon-secret" not found
Sep 14 01:33:18.368306 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:33:18.368257    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-56cb6897db-dvtww" podUID="526bc8e9-328b-4c29-912e-1e092a8b97fa"
Sep 14 01:33:18.384591 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:33:18.384559    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r" podUID="9935e3f9-36eb-4f10-9022-9a2699cf0866"
Sep 14 01:33:18.432128 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:33:18.432077    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jxz4h" podUID="248acebf-b6bf-407e-ad54-0976487a1b2c"
Sep 14 01:33:18.531002 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:33:18.530967    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:33:18.531002 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:33:18.530996    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:33:18.531168 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:33:18.531002    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:33:19.068802 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:33:19.068763    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:33:23.377538 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:33:23.377492    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:33:23.378082 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:33:23.377567    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xhr5r\" (UID: \"9935e3f9-36eb-4f10-9022-9a2699cf0866\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:33:23.378082 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:33:23.377595    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:33:23.378082 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:33:23.377641    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 01:33:23.378082 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:33:23.377663    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56cb6897db-dvtww: secret "image-registry-tls" not found
Sep 14 01:33:23.378082 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:33:23.377678    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 01:33:23.378082 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:33:23.377692    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 14 01:33:23.378082 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:33:23.377733    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls podName:526bc8e9-328b-4c29-912e-1e092a8b97fa nodeName:}" failed. No retries permitted until 2026-09-14 01:35:25.377715232 +0000 UTC m=+284.858166031 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls") pod "image-registry-56cb6897db-dvtww" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa") : secret "image-registry-tls" not found
Sep 14 01:33:23.378082 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:33:23.377746    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls podName:248acebf-b6bf-407e-ad54-0976487a1b2c nodeName:}" failed. No retries permitted until 2026-09-14 01:35:25.37774076 +0000 UTC m=+284.858191602 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls") pod "dns-default-jxz4h" (UID: "248acebf-b6bf-407e-ad54-0976487a1b2c") : secret "dns-default-metrics-tls" not found
Sep 14 01:33:23.378082 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:33:23.377756    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert podName:9935e3f9-36eb-4f10-9022-9a2699cf0866 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:25.377751413 +0000 UTC m=+284.858202212 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-xhr5r" (UID: "9935e3f9-36eb-4f10-9022-9a2699cf0866") : secret "networking-console-plugin-cert" not found
Sep 14 01:33:31.049760 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:33:31.049720    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:34:53.018348 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:34:53.018287    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:34:53.018864 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:34:53.018442    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 14 01:34:53.018864 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:34:53.018516    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs podName:ff2b27ad-c198-489c-9bb8-befeea447a64 nodeName:}" failed. No retries permitted until 2026-09-14 01:36:55.018499137 +0000 UTC m=+374.498949936 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs") pod "network-metrics-daemon-fv6nk" (UID: "ff2b27ad-c198-489c-9bb8-befeea447a64") : secret "metrics-daemon-secret" not found
Sep 14 01:35:04.568499 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.568416    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98"]
Sep 14 01:35:04.571204 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.571187    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98"
Sep 14 01:35:04.573152 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.573127    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-t6qgc"]
Sep 14 01:35:04.575775 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.575753    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"]
Sep 14 01:35:04.575904 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.575860    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6qgc"
Sep 14 01:35:04.577227 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.577192    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 14 01:35:04.577227 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.577186    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 14 01:35:04.577396 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.577283    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 14 01:35:04.577396 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.577383    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 14 01:35:04.577877 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.577863    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-mhlnh\""
Sep 14 01:35:04.577991 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.577892    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-tfbzn\""
Sep 14 01:35:04.578413 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.578395    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-84988c94c6-fp8j5"]
Sep 14 01:35:04.578519 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.578502    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:04.582518 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.582492    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 14 01:35:04.583939 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.583901    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-28pk5\""
Sep 14 01:35:04.584242 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.584220    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 14 01:35:04.584337 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.584292    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 14 01:35:04.585054 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.585032    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 14 01:35:04.585626 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.585608    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:04.587807 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.587783    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 14 01:35:04.588123 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.588104    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nwlf9\""
Sep 14 01:35:04.588616 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.588596    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 14 01:35:04.588816 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.588798    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 14 01:35:04.588954 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.588933    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 14 01:35:04.589052 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.588933    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 14 01:35:04.589144 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.588932    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 14 01:35:04.590207 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.590186    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98"]
Sep 14 01:35:04.591115 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.591095    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"]
Sep 14 01:35:04.591839 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.591822    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-t6qgc"]
Sep 14 01:35:04.597628 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.597608    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-84988c94c6-fp8j5"]
Sep 14 01:35:04.669144 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.669112    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-crc4t"]
Sep 14 01:35:04.672147 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.672130    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.674351 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.674329    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-pxq7p\""
Sep 14 01:35:04.674506 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.674373    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 14 01:35:04.674506 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.674449    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 14 01:35:04.674506 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.674502    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 14 01:35:04.674676 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.674530    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 14 01:35:04.679372 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.679351    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 14 01:35:04.681778 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.681758    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-crc4t"]
Sep 14 01:35:04.697044 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.697020    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r5l49\" (UID: \"63e51821-9401-42bc-8ecc-db50bae06d09\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:04.697162 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.697066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/066d2d8c-90ad-48fd-bd89-941f4c0e3c8c-serving-cert\") pod \"service-ca-operator-cf7c776b7-fqt98\" (UID: \"066d2d8c-90ad-48fd-bd89-941f4c0e3c8c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98"
Sep 14 01:35:04.697162 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.697085    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmr8z\" (UniqueName: \"kubernetes.io/projected/066d2d8c-90ad-48fd-bd89-941f4c0e3c8c-kube-api-access-jmr8z\") pod \"service-ca-operator-cf7c776b7-fqt98\" (UID: \"066d2d8c-90ad-48fd-bd89-941f4c0e3c8c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98"
Sep 14 01:35:04.697162 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.697141    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhvjv\" (UniqueName: \"kubernetes.io/projected/ec2956f5-bbef-4b16-bdb6-6f70058685b2-kube-api-access-vhvjv\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:04.697262 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.697174    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:04.697262 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.697196    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/63e51821-9401-42bc-8ecc-db50bae06d09-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-r5l49\" (UID: \"63e51821-9401-42bc-8ecc-db50bae06d09\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:04.697262 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.697241    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-stats-auth\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:04.697348 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.697261    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-default-certificate\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:04.697348 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.697296    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9dh7\" (UniqueName: \"kubernetes.io/projected/a6256eb6-c286-4a04-802a-2b69779f5e53-kube-api-access-b9dh7\") pod \"network-check-source-6f8b768455-t6qgc\" (UID: \"a6256eb6-c286-4a04-802a-2b69779f5e53\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6qgc"
Sep 14 01:35:04.697348 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.697319    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhff2\" (UniqueName: \"kubernetes.io/projected/63e51821-9401-42bc-8ecc-db50bae06d09-kube-api-access-hhff2\") pod \"cluster-monitoring-operator-6cd954b897-r5l49\" (UID: \"63e51821-9401-42bc-8ecc-db50bae06d09\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:04.697348 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.697339    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:04.697463 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.697375    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/066d2d8c-90ad-48fd-bd89-941f4c0e3c8c-config\") pod \"service-ca-operator-cf7c776b7-fqt98\" (UID: \"066d2d8c-90ad-48fd-bd89-941f4c0e3c8c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98"
Sep 14 01:35:04.797737 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.797704    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmr8z\" (UniqueName: \"kubernetes.io/projected/066d2d8c-90ad-48fd-bd89-941f4c0e3c8c-kube-api-access-jmr8z\") pod \"service-ca-operator-cf7c776b7-fqt98\" (UID: \"066d2d8c-90ad-48fd-bd89-941f4c0e3c8c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98"
Sep 14 01:35:04.797737 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.797740    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-stats-auth\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:04.798033 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.797763    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/624bd393-bc43-4a14-a0de-cf8ba209b0ed-serving-cert\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.798033 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.797783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-default-certificate\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:04.798033 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.797904    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/624bd393-bc43-4a14-a0de-cf8ba209b0ed-snapshots\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.798033 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.797966    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/066d2d8c-90ad-48fd-bd89-941f4c0e3c8c-config\") pod \"service-ca-operator-cf7c776b7-fqt98\" (UID: \"066d2d8c-90ad-48fd-bd89-941f4c0e3c8c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98"
Sep 14 01:35:04.798033 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.797993    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/624bd393-bc43-4a14-a0de-cf8ba209b0ed-tmp\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.798033 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.798021    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/624bd393-bc43-4a14-a0de-cf8ba209b0ed-trusted-ca-bundle\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.798271 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.798056    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r5l49\" (UID: \"63e51821-9401-42bc-8ecc-db50bae06d09\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:04.798271 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.798095    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/066d2d8c-90ad-48fd-bd89-941f4c0e3c8c-serving-cert\") pod \"service-ca-operator-cf7c776b7-fqt98\" (UID: \"066d2d8c-90ad-48fd-bd89-941f4c0e3c8c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98"
Sep 14 01:35:04.798271 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.798173    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:04.798271 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.798239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhvjv\" (UniqueName: \"kubernetes.io/projected/ec2956f5-bbef-4b16-bdb6-6f70058685b2-kube-api-access-vhvjv\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:04.798271 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:04.798183    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 01:35:04.798466 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.798285    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/624bd393-bc43-4a14-a0de-cf8ba209b0ed-service-ca-bundle\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.798466 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.798313    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/63e51821-9401-42bc-8ecc-db50bae06d09-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-r5l49\" (UID: \"63e51821-9401-42bc-8ecc-db50bae06d09\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:04.798466 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.798339    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxllq\" (UniqueName: \"kubernetes.io/projected/624bd393-bc43-4a14-a0de-cf8ba209b0ed-kube-api-access-cxllq\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.798466 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.798396    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:04.798466 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.798421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9dh7\" (UniqueName: \"kubernetes.io/projected/a6256eb6-c286-4a04-802a-2b69779f5e53-kube-api-access-b9dh7\") pod \"network-check-source-6f8b768455-t6qgc\" (UID: \"a6256eb6-c286-4a04-802a-2b69779f5e53\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6qgc"
Sep 14 01:35:04.798466 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.798447    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhff2\" (UniqueName: \"kubernetes.io/projected/63e51821-9401-42bc-8ecc-db50bae06d09-kube-api-access-hhff2\") pod \"cluster-monitoring-operator-6cd954b897-r5l49\" (UID: \"63e51821-9401-42bc-8ecc-db50bae06d09\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:04.798814 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:04.798253    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 01:35:04.798814 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.798524    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/066d2d8c-90ad-48fd-bd89-941f4c0e3c8c-config\") pod \"service-ca-operator-cf7c776b7-fqt98\" (UID: \"066d2d8c-90ad-48fd-bd89-941f4c0e3c8c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98"
Sep 14 01:35:04.798814 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:04.798567    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs podName:ec2956f5-bbef-4b16-bdb6-6f70058685b2 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:05.298547418 +0000 UTC m=+264.778998218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs") pod "router-default-84988c94c6-fp8j5" (UID: "ec2956f5-bbef-4b16-bdb6-6f70058685b2") : secret "router-metrics-certs-default" not found
Sep 14 01:35:04.798814 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:04.798607    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls podName:63e51821-9401-42bc-8ecc-db50bae06d09 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:05.298588186 +0000 UTC m=+264.779038996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-r5l49" (UID: "63e51821-9401-42bc-8ecc-db50bae06d09") : secret "cluster-monitoring-operator-tls" not found
Sep 14 01:35:04.798814 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:04.798664    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle podName:ec2956f5-bbef-4b16-bdb6-6f70058685b2 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:05.298650057 +0000 UTC m=+264.779100858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle") pod "router-default-84988c94c6-fp8j5" (UID: "ec2956f5-bbef-4b16-bdb6-6f70058685b2") : configmap references non-existent config key: service-ca.crt
Sep 14 01:35:04.799209 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.799188    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/63e51821-9401-42bc-8ecc-db50bae06d09-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-r5l49\" (UID: \"63e51821-9401-42bc-8ecc-db50bae06d09\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:04.800299 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.800279    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/066d2d8c-90ad-48fd-bd89-941f4c0e3c8c-serving-cert\") pod \"service-ca-operator-cf7c776b7-fqt98\" (UID: \"066d2d8c-90ad-48fd-bd89-941f4c0e3c8c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98"
Sep 14 01:35:04.800475 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.800457    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-default-certificate\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:04.800571 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.800547    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-stats-auth\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:04.806697 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.806671    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9dh7\" (UniqueName: \"kubernetes.io/projected/a6256eb6-c286-4a04-802a-2b69779f5e53-kube-api-access-b9dh7\") pod \"network-check-source-6f8b768455-t6qgc\" (UID: \"a6256eb6-c286-4a04-802a-2b69779f5e53\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6qgc"
Sep 14 01:35:04.807044 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.807028    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhff2\" (UniqueName: \"kubernetes.io/projected/63e51821-9401-42bc-8ecc-db50bae06d09-kube-api-access-hhff2\") pod \"cluster-monitoring-operator-6cd954b897-r5l49\" (UID: \"63e51821-9401-42bc-8ecc-db50bae06d09\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:04.807250 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.807230    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhvjv\" (UniqueName: \"kubernetes.io/projected/ec2956f5-bbef-4b16-bdb6-6f70058685b2-kube-api-access-vhvjv\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:04.807552 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.807535    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmr8z\" (UniqueName: \"kubernetes.io/projected/066d2d8c-90ad-48fd-bd89-941f4c0e3c8c-kube-api-access-jmr8z\") pod \"service-ca-operator-cf7c776b7-fqt98\" (UID: \"066d2d8c-90ad-48fd-bd89-941f4c0e3c8c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98"
Sep 14 01:35:04.888059 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.887967    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98"
Sep 14 01:35:04.891085 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.891053    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6qgc"
Sep 14 01:35:04.899188 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.899168    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/624bd393-bc43-4a14-a0de-cf8ba209b0ed-trusted-ca-bundle\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.899380 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.899356    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/624bd393-bc43-4a14-a0de-cf8ba209b0ed-service-ca-bundle\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.899442 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.899407    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxllq\" (UniqueName: \"kubernetes.io/projected/624bd393-bc43-4a14-a0de-cf8ba209b0ed-kube-api-access-cxllq\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.899611 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.899508    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/624bd393-bc43-4a14-a0de-cf8ba209b0ed-serving-cert\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.899611 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.899557    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/624bd393-bc43-4a14-a0de-cf8ba209b0ed-snapshots\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.899611 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.899586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/624bd393-bc43-4a14-a0de-cf8ba209b0ed-tmp\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.899980 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.899903    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/624bd393-bc43-4a14-a0de-cf8ba209b0ed-service-ca-bundle\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.900066 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.900027    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/624bd393-bc43-4a14-a0de-cf8ba209b0ed-tmp\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.900216 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.900172    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/624bd393-bc43-4a14-a0de-cf8ba209b0ed-trusted-ca-bundle\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.900650 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.900521    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/624bd393-bc43-4a14-a0de-cf8ba209b0ed-snapshots\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.902524 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.902504    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/624bd393-bc43-4a14-a0de-cf8ba209b0ed-serving-cert\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.908409 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.908388    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxllq\" (UniqueName: \"kubernetes.io/projected/624bd393-bc43-4a14-a0de-cf8ba209b0ed-kube-api-access-cxllq\") pod \"insights-operator-5844b856db-crc4t\" (UID: \"624bd393-bc43-4a14-a0de-cf8ba209b0ed\") " pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:04.982321 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:04.982290    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-crc4t"
Sep 14 01:35:05.166009 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:05.165925    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-t6qgc"]
Sep 14 01:35:05.168933 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:05.168886    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98"]
Sep 14 01:35:05.169227 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:05.169199    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6256eb6_c286_4a04_802a_2b69779f5e53.slice/crio-90ab1c80b03070f560e9ca65cbe0a26dbcb691e387ebbfa3696ea169ca98e025 WatchSource:0}: Error finding container 90ab1c80b03070f560e9ca65cbe0a26dbcb691e387ebbfa3696ea169ca98e025: Status 404 returned error can't find the container with id 90ab1c80b03070f560e9ca65cbe0a26dbcb691e387ebbfa3696ea169ca98e025
Sep 14 01:35:05.171737 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:05.171693    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod066d2d8c_90ad_48fd_bd89_941f4c0e3c8c.slice/crio-7033bde5e2b363d5e83c048b685058acdecb5cb16882bf08b417ca302fc9bc2b WatchSource:0}: Error finding container 7033bde5e2b363d5e83c048b685058acdecb5cb16882bf08b417ca302fc9bc2b: Status 404 returned error can't find the container with id 7033bde5e2b363d5e83c048b685058acdecb5cb16882bf08b417ca302fc9bc2b
Sep 14 01:35:05.249347 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:05.249306    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-crc4t"]
Sep 14 01:35:05.252437 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:05.252410    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod624bd393_bc43_4a14_a0de_cf8ba209b0ed.slice/crio-b675bcf0930fae5df3ad1a4efa9b063598a822d2bc4950ea927ff10947803c90 WatchSource:0}: Error finding container b675bcf0930fae5df3ad1a4efa9b063598a822d2bc4950ea927ff10947803c90: Status 404 returned error can't find the container with id b675bcf0930fae5df3ad1a4efa9b063598a822d2bc4950ea927ff10947803c90
Sep 14 01:35:05.303497 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:05.303452    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r5l49\" (UID: \"63e51821-9401-42bc-8ecc-db50bae06d09\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:05.303672 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:05.303518    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:05.303672 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:05.303591    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:05.303672 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:05.303594    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 01:35:05.303672 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:05.303652    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 01:35:05.303864 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:05.303665    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls podName:63e51821-9401-42bc-8ecc-db50bae06d09 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:06.303644429 +0000 UTC m=+265.784095227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-r5l49" (UID: "63e51821-9401-42bc-8ecc-db50bae06d09") : secret "cluster-monitoring-operator-tls" not found
Sep 14 01:35:05.303864 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:05.303719    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs podName:ec2956f5-bbef-4b16-bdb6-6f70058685b2 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:06.30370517 +0000 UTC m=+265.784155972 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs") pod "router-default-84988c94c6-fp8j5" (UID: "ec2956f5-bbef-4b16-bdb6-6f70058685b2") : secret "router-metrics-certs-default" not found
Sep 14 01:35:05.303864 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:05.303743    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle podName:ec2956f5-bbef-4b16-bdb6-6f70058685b2 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:06.303733299 +0000 UTC m=+265.784184111 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle") pod "router-default-84988c94c6-fp8j5" (UID: "ec2956f5-bbef-4b16-bdb6-6f70058685b2") : configmap references non-existent config key: service-ca.crt
Sep 14 01:35:05.788110 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:05.788074    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-crc4t" event={"ID":"624bd393-bc43-4a14-a0de-cf8ba209b0ed","Type":"ContainerStarted","Data":"b675bcf0930fae5df3ad1a4efa9b063598a822d2bc4950ea927ff10947803c90"}
Sep 14 01:35:05.790255 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:05.790215    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6qgc" event={"ID":"a6256eb6-c286-4a04-802a-2b69779f5e53","Type":"ContainerStarted","Data":"4215ae3b98e891088437e290f8d9e51c0912734ec1687db8faebb1ecf4889fe0"}
Sep 14 01:35:05.790395 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:05.790314    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6qgc" event={"ID":"a6256eb6-c286-4a04-802a-2b69779f5e53","Type":"ContainerStarted","Data":"90ab1c80b03070f560e9ca65cbe0a26dbcb691e387ebbfa3696ea169ca98e025"}
Sep 14 01:35:05.796166 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:05.796135    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98" event={"ID":"066d2d8c-90ad-48fd-bd89-941f4c0e3c8c","Type":"ContainerStarted","Data":"7033bde5e2b363d5e83c048b685058acdecb5cb16882bf08b417ca302fc9bc2b"}
Sep 14 01:35:05.807048 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:05.806994    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-t6qgc" podStartSLOduration=1.806976554 podStartE2EDuration="1.806976554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:35:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:05.805482467 +0000 UTC m=+265.285933288" watchObservedRunningTime="2026-09-14 01:35:05.806976554 +0000 UTC m=+265.287427378"
Sep 14 01:35:06.312226 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:06.312181    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:06.312483 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:06.312273    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:06.312483 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:06.312330    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r5l49\" (UID: \"63e51821-9401-42bc-8ecc-db50bae06d09\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:06.312483 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:06.312365    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 01:35:06.312483 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:06.312438    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 01:35:06.312483 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:06.312451    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs podName:ec2956f5-bbef-4b16-bdb6-6f70058685b2 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:08.312430614 +0000 UTC m=+267.792881433 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs") pod "router-default-84988c94c6-fp8j5" (UID: "ec2956f5-bbef-4b16-bdb6-6f70058685b2") : secret "router-metrics-certs-default" not found
Sep 14 01:35:06.312720 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:06.312483    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle podName:ec2956f5-bbef-4b16-bdb6-6f70058685b2 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:08.312464896 +0000 UTC m=+267.792915708 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle") pod "router-default-84988c94c6-fp8j5" (UID: "ec2956f5-bbef-4b16-bdb6-6f70058685b2") : configmap references non-existent config key: service-ca.crt
Sep 14 01:35:06.312720 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:06.312509    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls podName:63e51821-9401-42bc-8ecc-db50bae06d09 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:08.312499172 +0000 UTC m=+267.792949977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-r5l49" (UID: "63e51821-9401-42bc-8ecc-db50bae06d09") : secret "cluster-monitoring-operator-tls" not found
Sep 14 01:35:07.806507 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:07.806466    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98" event={"ID":"066d2d8c-90ad-48fd-bd89-941f4c0e3c8c","Type":"ContainerStarted","Data":"79cb8a67b0c03ea6ecdf18502ca697acd5219e1b99cd1981e083e740b3a78128"}
Sep 14 01:35:07.808131 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:07.808106    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-crc4t" event={"ID":"624bd393-bc43-4a14-a0de-cf8ba209b0ed","Type":"ContainerStarted","Data":"3b997a86fb00fb4492b1c07206015750c02a7e53b42f7022f59b5d05cdde75f5"}
Sep 14 01:35:07.820708 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:07.820633    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98" podStartSLOduration=1.377279789 podStartE2EDuration="3.820613392s" totalImagesPullingTime="2.443333603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:35:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:07.81930641 +0000 UTC m=+267.299757236" watchObservedRunningTime="2026-09-14 01:35:07.820613392 +0000 UTC m=+267.301064214"
Sep 14 01:35:07.833509 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:07.833456    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-crc4t" podStartSLOduration=1.469605038 podStartE2EDuration="3.833440317s" totalImagesPullingTime="2.363835279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:35:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:07.832071158 +0000 UTC m=+267.312521980" watchObservedRunningTime="2026-09-14 01:35:07.833440317 +0000 UTC m=+267.313891137"
Sep 14 01:35:08.333328 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.333288    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:08.333490 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.333353    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r5l49\" (UID: \"63e51821-9401-42bc-8ecc-db50bae06d09\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:08.333490 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.333381    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:08.333490 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:08.333481    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 01:35:08.333595 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:08.333487    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 01:35:08.333595 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:08.333532    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle podName:ec2956f5-bbef-4b16-bdb6-6f70058685b2 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:12.333507029 +0000 UTC m=+271.813957845 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle") pod "router-default-84988c94c6-fp8j5" (UID: "ec2956f5-bbef-4b16-bdb6-6f70058685b2") : configmap references non-existent config key: service-ca.crt
Sep 14 01:35:08.333595 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:08.333560    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs podName:ec2956f5-bbef-4b16-bdb6-6f70058685b2 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:12.333549615 +0000 UTC m=+271.814000422 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs") pod "router-default-84988c94c6-fp8j5" (UID: "ec2956f5-bbef-4b16-bdb6-6f70058685b2") : secret "router-metrics-certs-default" not found
Sep 14 01:35:08.333595 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:08.333578    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls podName:63e51821-9401-42bc-8ecc-db50bae06d09 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:12.3335685 +0000 UTC m=+271.814019304 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-r5l49" (UID: "63e51821-9401-42bc-8ecc-db50bae06d09") : secret "cluster-monitoring-operator-tls" not found
Sep 14 01:35:08.434922 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.434887    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2cl5j"]
Sep 14 01:35:08.438311 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.438287    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:08.440541 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.440518    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 14 01:35:08.440981 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.440962    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 14 01:35:08.441082 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.441006    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dqk8k\""
Sep 14 01:35:08.451394 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.451370    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2cl5j"]
Sep 14 01:35:08.535409 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.535372    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:08.535409 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.535405    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvbcm\" (UniqueName: \"kubernetes.io/projected/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-api-access-zvbcm\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:08.535673 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.535540    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:08.535673 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.535578    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-data-volume\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:08.535673 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.535617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-crio-socket\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:08.636596 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.636504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:08.636596 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.636542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvbcm\" (UniqueName: \"kubernetes.io/projected/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-api-access-zvbcm\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:08.636596 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.636598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:08.636873 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.636616    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-data-volume\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:08.636873 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.636635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-crio-socket\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:08.636873 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:08.636663    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 01:35:08.636873 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:08.636693    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 01:35:08.636873 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.636734    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-crio-socket\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:08.636873 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:08.636739    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls podName:d062c861-cb50-4bdd-bac9-7eba1d0a74bd nodeName:}" failed. No retries permitted until 2026-09-14 01:35:09.136717109 +0000 UTC m=+268.617167911 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cl5j" (UID: "d062c861-cb50-4bdd-bac9-7eba1d0a74bd") : secret "insights-runtime-extractor-tls" not found
Sep 14 01:35:08.636873 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:08.636758    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm podName:d062c861-cb50-4bdd-bac9-7eba1d0a74bd nodeName:}" failed. No retries permitted until 2026-09-14 01:35:09.136748926 +0000 UTC m=+268.617199726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cl5j" (UID: "d062c861-cb50-4bdd-bac9-7eba1d0a74bd") : configmap "kube-rbac-proxy" not found
Sep 14 01:35:08.637126 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.636965    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-data-volume\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:08.643707 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:08.643685    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvbcm\" (UniqueName: \"kubernetes.io/projected/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-api-access-zvbcm\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:09.141489 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:09.141453    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:09.141904 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:09.141533    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:09.141904 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:09.141609    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 01:35:09.141904 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:09.141670    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 01:35:09.141904 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:09.141690    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls podName:d062c861-cb50-4bdd-bac9-7eba1d0a74bd nodeName:}" failed. No retries permitted until 2026-09-14 01:35:10.141671284 +0000 UTC m=+269.622122084 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cl5j" (UID: "d062c861-cb50-4bdd-bac9-7eba1d0a74bd") : secret "insights-runtime-extractor-tls" not found
Sep 14 01:35:09.141904 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:09.141724    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm podName:d062c861-cb50-4bdd-bac9-7eba1d0a74bd nodeName:}" failed. No retries permitted until 2026-09-14 01:35:10.141710273 +0000 UTC m=+269.622161072 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cl5j" (UID: "d062c861-cb50-4bdd-bac9-7eba1d0a74bd") : configmap "kube-rbac-proxy" not found
Sep 14 01:35:10.150022 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:10.149984    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:10.150518 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:10.150067    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:10.150518 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:10.150130    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 01:35:10.150518 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:10.150180    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm podName:d062c861-cb50-4bdd-bac9-7eba1d0a74bd nodeName:}" failed. No retries permitted until 2026-09-14 01:35:12.15016657 +0000 UTC m=+271.630617369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cl5j" (UID: "d062c861-cb50-4bdd-bac9-7eba1d0a74bd") : configmap "kube-rbac-proxy" not found
Sep 14 01:35:10.150518 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:10.150129    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 01:35:10.150518 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:10.150262    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls podName:d062c861-cb50-4bdd-bac9-7eba1d0a74bd nodeName:}" failed. No retries permitted until 2026-09-14 01:35:12.150247258 +0000 UTC m=+271.630698074 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cl5j" (UID: "d062c861-cb50-4bdd-bac9-7eba1d0a74bd") : secret "insights-runtime-extractor-tls" not found
Sep 14 01:35:10.366056 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:10.366025    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bzb4b_4ab20462-366d-4e07-a302-25be0aef9dd9/dns-node-resolver/0.log"
Sep 14 01:35:10.891806 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:10.891775    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-598dz"]
Sep 14 01:35:10.895034 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:10.895017    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-598dz"
Sep 14 01:35:10.896833 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:10.896807    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 14 01:35:10.897597 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:10.897575    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 14 01:35:10.897597 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:10.897591    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 14 01:35:10.897758 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:10.897575    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-5ndp7\""
Sep 14 01:35:10.897758 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:10.897622    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 14 01:35:10.903372 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:10.903339    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-598dz"]
Sep 14 01:35:11.059474 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.059440    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhl7s\" (UniqueName: \"kubernetes.io/projected/967892ad-2e35-44bf-82a6-6c08b3eaad9b-kube-api-access-nhl7s\") pod \"service-ca-568869fdc5-598dz\" (UID: \"967892ad-2e35-44bf-82a6-6c08b3eaad9b\") " pod="openshift-service-ca/service-ca-568869fdc5-598dz"
Sep 14 01:35:11.059642 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.059573    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/967892ad-2e35-44bf-82a6-6c08b3eaad9b-signing-cabundle\") pod \"service-ca-568869fdc5-598dz\" (UID: \"967892ad-2e35-44bf-82a6-6c08b3eaad9b\") " pod="openshift-service-ca/service-ca-568869fdc5-598dz"
Sep 14 01:35:11.059702 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.059647    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/967892ad-2e35-44bf-82a6-6c08b3eaad9b-signing-key\") pod \"service-ca-568869fdc5-598dz\" (UID: \"967892ad-2e35-44bf-82a6-6c08b3eaad9b\") " pod="openshift-service-ca/service-ca-568869fdc5-598dz"
Sep 14 01:35:11.160265 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.160185    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/967892ad-2e35-44bf-82a6-6c08b3eaad9b-signing-key\") pod \"service-ca-568869fdc5-598dz\" (UID: \"967892ad-2e35-44bf-82a6-6c08b3eaad9b\") " pod="openshift-service-ca/service-ca-568869fdc5-598dz"
Sep 14 01:35:11.160265 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.160256    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhl7s\" (UniqueName: \"kubernetes.io/projected/967892ad-2e35-44bf-82a6-6c08b3eaad9b-kube-api-access-nhl7s\") pod \"service-ca-568869fdc5-598dz\" (UID: \"967892ad-2e35-44bf-82a6-6c08b3eaad9b\") " pod="openshift-service-ca/service-ca-568869fdc5-598dz"
Sep 14 01:35:11.160748 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.160344    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/967892ad-2e35-44bf-82a6-6c08b3eaad9b-signing-cabundle\") pod \"service-ca-568869fdc5-598dz\" (UID: \"967892ad-2e35-44bf-82a6-6c08b3eaad9b\") " pod="openshift-service-ca/service-ca-568869fdc5-598dz"
Sep 14 01:35:11.160980 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.160960    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/967892ad-2e35-44bf-82a6-6c08b3eaad9b-signing-cabundle\") pod \"service-ca-568869fdc5-598dz\" (UID: \"967892ad-2e35-44bf-82a6-6c08b3eaad9b\") " pod="openshift-service-ca/service-ca-568869fdc5-598dz"
Sep 14 01:35:11.162639 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.162619    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/967892ad-2e35-44bf-82a6-6c08b3eaad9b-signing-key\") pod \"service-ca-568869fdc5-598dz\" (UID: \"967892ad-2e35-44bf-82a6-6c08b3eaad9b\") " pod="openshift-service-ca/service-ca-568869fdc5-598dz"
Sep 14 01:35:11.167564 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.167533    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhl7s\" (UniqueName: \"kubernetes.io/projected/967892ad-2e35-44bf-82a6-6c08b3eaad9b-kube-api-access-nhl7s\") pod \"service-ca-568869fdc5-598dz\" (UID: \"967892ad-2e35-44bf-82a6-6c08b3eaad9b\") " pod="openshift-service-ca/service-ca-568869fdc5-598dz"
Sep 14 01:35:11.204678 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.204642    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-598dz"
Sep 14 01:35:11.472310 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.472274    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-598dz"]
Sep 14 01:35:11.475836 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:11.475812    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod967892ad_2e35_44bf_82a6_6c08b3eaad9b.slice/crio-51e00d795a4095ff4f56cba4109c22a58830f3bfa10349c50afa5c088c57466c WatchSource:0}: Error finding container 51e00d795a4095ff4f56cba4109c22a58830f3bfa10349c50afa5c088c57466c: Status 404 returned error can't find the container with id 51e00d795a4095ff4f56cba4109c22a58830f3bfa10349c50afa5c088c57466c
Sep 14 01:35:11.567058 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.567028    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rlnvn_f00644fe-4d6e-49e3-a485-eb4029ad78ad/node-ca/0.log"
Sep 14 01:35:11.822334 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.822249    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-598dz" event={"ID":"967892ad-2e35-44bf-82a6-6c08b3eaad9b","Type":"ContainerStarted","Data":"b2558f45627ec7f326780a3e1624517c6e0dcc613c5cea66c29e1ecc13506bba"}
Sep 14 01:35:11.822334 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.822296    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-598dz" event={"ID":"967892ad-2e35-44bf-82a6-6c08b3eaad9b","Type":"ContainerStarted","Data":"51e00d795a4095ff4f56cba4109c22a58830f3bfa10349c50afa5c088c57466c"}
Sep 14 01:35:11.836655 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.836529    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-598dz" podStartSLOduration=1.8365072759999999 podStartE2EDuration="1.836507276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:35:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:11.83639406 +0000 UTC m=+271.316844881" watchObservedRunningTime="2026-09-14 01:35:11.836507276 +0000 UTC m=+271.316958098"
Sep 14 01:35:11.966804 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:11.966769    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5fs9z_74829c0c-ae15-4b20-b653-1e23c44c899b/serve-healthcheck-canary/0.log"
Sep 14 01:35:12.170662 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:12.170580    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:12.171082 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:12.170728    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 01:35:12.171082 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:12.170802    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:12.171082 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:12.170812    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm podName:d062c861-cb50-4bdd-bac9-7eba1d0a74bd nodeName:}" failed. No retries permitted until 2026-09-14 01:35:16.170791304 +0000 UTC m=+275.651242109 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cl5j" (UID: "d062c861-cb50-4bdd-bac9-7eba1d0a74bd") : configmap "kube-rbac-proxy" not found
Sep 14 01:35:12.171082 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:12.170943    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 01:35:12.171082 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:12.170997    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls podName:d062c861-cb50-4bdd-bac9-7eba1d0a74bd nodeName:}" failed. No retries permitted until 2026-09-14 01:35:16.170983821 +0000 UTC m=+275.651434619 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cl5j" (UID: "d062c861-cb50-4bdd-bac9-7eba1d0a74bd") : secret "insights-runtime-extractor-tls" not found
Sep 14 01:35:12.373273 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:12.373234    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r5l49\" (UID: \"63e51821-9401-42bc-8ecc-db50bae06d09\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:12.373448 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:12.373282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:12.373448 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:12.373383    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 01:35:12.373448 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:12.373405    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 01:35:12.373560 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:12.373467    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls podName:63e51821-9401-42bc-8ecc-db50bae06d09 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:20.373451693 +0000 UTC m=+279.853902520 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-r5l49" (UID: "63e51821-9401-42bc-8ecc-db50bae06d09") : secret "cluster-monitoring-operator-tls" not found
Sep 14 01:35:12.373560 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:12.373493    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:12.373645 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:12.373563    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle podName:ec2956f5-bbef-4b16-bdb6-6f70058685b2 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:20.373551315 +0000 UTC m=+279.854002136 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle") pod "router-default-84988c94c6-fp8j5" (UID: "ec2956f5-bbef-4b16-bdb6-6f70058685b2") : configmap references non-existent config key: service-ca.crt
Sep 14 01:35:12.373645 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:12.373590    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs podName:ec2956f5-bbef-4b16-bdb6-6f70058685b2 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:20.373579436 +0000 UTC m=+279.854030236 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs") pod "router-default-84988c94c6-fp8j5" (UID: "ec2956f5-bbef-4b16-bdb6-6f70058685b2") : secret "router-metrics-certs-default" not found
Sep 14 01:35:16.209831 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:16.209793    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:16.210283 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:16.209957    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 01:35:16.210283 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:16.209960    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:16.210283 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:16.210035    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm podName:d062c861-cb50-4bdd-bac9-7eba1d0a74bd nodeName:}" failed. No retries permitted until 2026-09-14 01:35:24.210017737 +0000 UTC m=+283.690468555 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cl5j" (UID: "d062c861-cb50-4bdd-bac9-7eba1d0a74bd") : configmap "kube-rbac-proxy" not found
Sep 14 01:35:16.210283 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:16.210037    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 01:35:16.210283 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:16.210139    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls podName:d062c861-cb50-4bdd-bac9-7eba1d0a74bd nodeName:}" failed. No retries permitted until 2026-09-14 01:35:24.210123763 +0000 UTC m=+283.690574580 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cl5j" (UID: "d062c861-cb50-4bdd-bac9-7eba1d0a74bd") : secret "insights-runtime-extractor-tls" not found
Sep 14 01:35:20.446618 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:20.446580    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:20.447055 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:20.446671    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r5l49\" (UID: \"63e51821-9401-42bc-8ecc-db50bae06d09\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:20.447055 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:20.446697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:20.447379 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:20.447356    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec2956f5-bbef-4b16-bdb6-6f70058685b2-service-ca-bundle\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:20.449161 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:20.449137    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63e51821-9401-42bc-8ecc-db50bae06d09-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-r5l49\" (UID: \"63e51821-9401-42bc-8ecc-db50bae06d09\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:20.449293 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:20.449273    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec2956f5-bbef-4b16-bdb6-6f70058685b2-metrics-certs\") pod \"router-default-84988c94c6-fp8j5\" (UID: \"ec2956f5-bbef-4b16-bdb6-6f70058685b2\") " pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:20.498424 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:20.498386    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"
Sep 14 01:35:20.504729 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:20.504711    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:20.778171 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:20.778142    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49"]
Sep 14 01:35:20.781694 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:20.781654    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-84988c94c6-fp8j5"]
Sep 14 01:35:20.781804 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:20.781707    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63e51821_9401_42bc_8ecc_db50bae06d09.slice/crio-d2428b94354c12e3962e4c8d1d31b4af49854b85acec582865f9fa0ffad41167 WatchSource:0}: Error finding container d2428b94354c12e3962e4c8d1d31b4af49854b85acec582865f9fa0ffad41167: Status 404 returned error can't find the container with id d2428b94354c12e3962e4c8d1d31b4af49854b85acec582865f9fa0ffad41167
Sep 14 01:35:20.784461 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:20.784434    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec2956f5_bbef_4b16_bdb6_6f70058685b2.slice/crio-81defdd1c8e50628fc7c8a5cac615eaa43588d15e1e252d5e18262f2b4323db8 WatchSource:0}: Error finding container 81defdd1c8e50628fc7c8a5cac615eaa43588d15e1e252d5e18262f2b4323db8: Status 404 returned error can't find the container with id 81defdd1c8e50628fc7c8a5cac615eaa43588d15e1e252d5e18262f2b4323db8
Sep 14 01:35:20.850962 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:20.850928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49" event={"ID":"63e51821-9401-42bc-8ecc-db50bae06d09","Type":"ContainerStarted","Data":"d2428b94354c12e3962e4c8d1d31b4af49854b85acec582865f9fa0ffad41167"}
Sep 14 01:35:20.852598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:20.852576    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-84988c94c6-fp8j5" event={"ID":"ec2956f5-bbef-4b16-bdb6-6f70058685b2","Type":"ContainerStarted","Data":"3fdb3d92cb4a53ad2f13ab80588f1caeaee8f78758e944173535c604e4e772ad"}
Sep 14 01:35:20.852711 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:20.852621    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-84988c94c6-fp8j5" event={"ID":"ec2956f5-bbef-4b16-bdb6-6f70058685b2","Type":"ContainerStarted","Data":"81defdd1c8e50628fc7c8a5cac615eaa43588d15e1e252d5e18262f2b4323db8"}
Sep 14 01:35:20.871223 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:20.871169    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-84988c94c6-fp8j5" podStartSLOduration=16.871150919 podStartE2EDuration="16.871150919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:35:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:20.870027423 +0000 UTC m=+280.350478255" watchObservedRunningTime="2026-09-14 01:35:20.871150919 +0000 UTC m=+280.351601732"
Sep 14 01:35:21.505032 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:21.504956    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:21.507950 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:21.507906    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:21.531822 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:21.531781    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r" podUID="9935e3f9-36eb-4f10-9022-9a2699cf0866"
Sep 14 01:35:21.531822 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:21.531783    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jxz4h" podUID="248acebf-b6bf-407e-ad54-0976487a1b2c"
Sep 14 01:35:21.532069 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:21.531861    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-56cb6897db-dvtww" podUID="526bc8e9-328b-4c29-912e-1e092a8b97fa"
Sep 14 01:35:21.856210 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:21.856121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:35:21.856210 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:21.856121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:35:21.856210 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:21.856131    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:35:21.856992 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:21.856961    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:21.857986 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:21.857952    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-84988c94c6-fp8j5"
Sep 14 01:35:22.866874 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:22.866823    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49" event={"ID":"63e51821-9401-42bc-8ecc-db50bae06d09","Type":"ContainerStarted","Data":"5f0f5fe89dd4c6d69573d667e9fcbeb8692d95f639c8ab55a3eca01f91a49719"}
Sep 14 01:35:22.884136 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:22.884087    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-r5l49" podStartSLOduration=17.256524162 podStartE2EDuration="18.884066492s" totalImagesPullingTime="1.62754233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:35:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:22.882428559 +0000 UTC m=+282.362879382" watchObservedRunningTime="2026-09-14 01:35:22.884066492 +0000 UTC m=+282.364517314"
Sep 14 01:35:24.277488 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:24.277449    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:24.277870 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:24.277557    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:24.278031 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:24.278009    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:24.279949 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:24.279904    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d062c861-cb50-4bdd-bac9-7eba1d0a74bd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl5j\" (UID: \"d062c861-cb50-4bdd-bac9-7eba1d0a74bd\") " pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:24.351567 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:24.351531    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2cl5j"
Sep 14 01:35:24.617499 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:24.617419    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2cl5j"]
Sep 14 01:35:24.620178 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:24.620136    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd062c861_cb50_4bdd_bac9_7eba1d0a74bd.slice/crio-5cc97e9c866f4f75dd3c194618b32e554cd8c56723b3a0763fc23d13cbbc2df6 WatchSource:0}: Error finding container 5cc97e9c866f4f75dd3c194618b32e554cd8c56723b3a0763fc23d13cbbc2df6: Status 404 returned error can't find the container with id 5cc97e9c866f4f75dd3c194618b32e554cd8c56723b3a0763fc23d13cbbc2df6
Sep 14 01:35:24.874876 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:24.874796    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cl5j" event={"ID":"d062c861-cb50-4bdd-bac9-7eba1d0a74bd","Type":"ContainerStarted","Data":"6af386262316857ceeb598f3c58d9ab79bfddd43ac93981843f2a90f1f056d97"}
Sep 14 01:35:24.874876 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:24.874853    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cl5j" event={"ID":"d062c861-cb50-4bdd-bac9-7eba1d0a74bd","Type":"ContainerStarted","Data":"5cc97e9c866f4f75dd3c194618b32e554cd8c56723b3a0763fc23d13cbbc2df6"}
Sep 14 01:35:25.383968 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.383897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xhr5r\" (UID: \"9935e3f9-36eb-4f10-9022-9a2699cf0866\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:35:25.384346 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.383980    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:35:25.384346 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.384016    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:35:25.386486 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.386454    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/248acebf-b6bf-407e-ad54-0976487a1b2c-metrics-tls\") pod \"dns-default-jxz4h\" (UID: \"248acebf-b6bf-407e-ad54-0976487a1b2c\") " pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:35:25.386608 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.386535    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls\") pod \"image-registry-56cb6897db-dvtww\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") " pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:35:25.386608 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.386545    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9935e3f9-36eb-4f10-9022-9a2699cf0866-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xhr5r\" (UID: \"9935e3f9-36eb-4f10-9022-9a2699cf0866\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:35:25.459683 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.459643    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vh46f\""
Sep 14 01:35:25.459683 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.459649    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jt2tm\""
Sep 14 01:35:25.459683 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.459679    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jfmw6\""
Sep 14 01:35:25.467343 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.467324    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:35:25.467433 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.467420    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"
Sep 14 01:35:25.467473 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.467463    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:35:25.749805 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.749771    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r"]
Sep 14 01:35:25.754704 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.754683    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jxz4h"]
Sep 14 01:35:25.754850 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.754799    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56cb6897db-dvtww"]
Sep 14 01:35:25.755170 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:25.755147    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9935e3f9_36eb_4f10_9022_9a2699cf0866.slice/crio-194b9f6f32fb8a11af68c7cb93c1eef0b860c94d0958e73dd616b0384d0f0944 WatchSource:0}: Error finding container 194b9f6f32fb8a11af68c7cb93c1eef0b860c94d0958e73dd616b0384d0f0944: Status 404 returned error can't find the container with id 194b9f6f32fb8a11af68c7cb93c1eef0b860c94d0958e73dd616b0384d0f0944
Sep 14 01:35:25.756960 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:25.756751    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod248acebf_b6bf_407e_ad54_0976487a1b2c.slice/crio-329668074b34e578d1b6b10ae3875939ea6a8cf8b679c0db1419bcd532c8d814 WatchSource:0}: Error finding container 329668074b34e578d1b6b10ae3875939ea6a8cf8b679c0db1419bcd532c8d814: Status 404 returned error can't find the container with id 329668074b34e578d1b6b10ae3875939ea6a8cf8b679c0db1419bcd532c8d814
Sep 14 01:35:25.757864 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:25.757837    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod526bc8e9_328b_4c29_912e_1e092a8b97fa.slice/crio-e1dc05826bebd71c66c6b453bc6347db08b321c9c3f15f783dc05d61840f16fb WatchSource:0}: Error finding container e1dc05826bebd71c66c6b453bc6347db08b321c9c3f15f783dc05d61840f16fb: Status 404 returned error can't find the container with id e1dc05826bebd71c66c6b453bc6347db08b321c9c3f15f783dc05d61840f16fb
Sep 14 01:35:25.883081 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.883032    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jxz4h" event={"ID":"248acebf-b6bf-407e-ad54-0976487a1b2c","Type":"ContainerStarted","Data":"329668074b34e578d1b6b10ae3875939ea6a8cf8b679c0db1419bcd532c8d814"}
Sep 14 01:35:25.885196 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.885161    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56cb6897db-dvtww" event={"ID":"526bc8e9-328b-4c29-912e-1e092a8b97fa","Type":"ContainerStarted","Data":"1a790a673101e0acf089db196f967e32e7de8c1d278417abd7d0953cd13a3b38"}
Sep 14 01:35:25.885363 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.885226    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56cb6897db-dvtww" event={"ID":"526bc8e9-328b-4c29-912e-1e092a8b97fa","Type":"ContainerStarted","Data":"e1dc05826bebd71c66c6b453bc6347db08b321c9c3f15f783dc05d61840f16fb"}
Sep 14 01:35:25.885363 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.885332    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:35:25.887669 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.887370    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cl5j" event={"ID":"d062c861-cb50-4bdd-bac9-7eba1d0a74bd","Type":"ContainerStarted","Data":"1d6f5655dd1dec761a6b5dc6f2aaaa95a7d71e81d628fc29086a38c38ab9d18d"}
Sep 14 01:35:25.889298 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.889271    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r" event={"ID":"9935e3f9-36eb-4f10-9022-9a2699cf0866","Type":"ContainerStarted","Data":"194b9f6f32fb8a11af68c7cb93c1eef0b860c94d0958e73dd616b0384d0f0944"}
Sep 14 01:35:25.901599 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:25.901546    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56cb6897db-dvtww" podStartSLOduration=284.901528456 podStartE2EDuration="4m44.901528456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:30:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:25.900933019 +0000 UTC m=+285.381383840" watchObservedRunningTime="2026-09-14 01:35:25.901528456 +0000 UTC m=+285.381979277"
Sep 14 01:35:26.899395 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:26.899297    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cl5j" event={"ID":"d062c861-cb50-4bdd-bac9-7eba1d0a74bd","Type":"ContainerStarted","Data":"74ab088ae21c0d456e0a8f629e415ce44ee6a1a59dee1c30625b029e19daa77a"}
Sep 14 01:35:26.914759 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:26.914702    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2cl5j" podStartSLOduration=17.146522802 podStartE2EDuration="18.914679706s" totalImagesPullingTime="1.768156904s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 01:35:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:26.9140189 +0000 UTC m=+286.394469722" watchObservedRunningTime="2026-09-14 01:35:26.914679706 +0000 UTC m=+286.395130521"
Sep 14 01:35:27.905190 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:27.905143    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r" event={"ID":"9935e3f9-36eb-4f10-9022-9a2699cf0866","Type":"ContainerStarted","Data":"29e2246d2f4acc0fa81e6f1b8309a2d2cc85272dbabd695dc6e5bc64c51aa71d"}
Sep 14 01:35:27.919346 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:27.919247    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xhr5r" podStartSLOduration=267.430403162 podStartE2EDuration="4m28.919228152s" totalImagesPullingTime="1.48882499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:30:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:27.918640721 +0000 UTC m=+287.399091543" watchObservedRunningTime="2026-09-14 01:35:27.919228152 +0000 UTC m=+287.399678974"
Sep 14 01:35:28.910610 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:28.910530    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jxz4h" event={"ID":"248acebf-b6bf-407e-ad54-0976487a1b2c","Type":"ContainerStarted","Data":"405a39ec0f9621fff5585e47577b7d82df7197b994bb2e756ff2496bb1c75a7e"}
Sep 14 01:35:28.910610 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:28.910585    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jxz4h" event={"ID":"248acebf-b6bf-407e-ad54-0976487a1b2c","Type":"ContainerStarted","Data":"367238451ac4f6b52ee7714458139483c959430e0c703348be45c57dfd9839d5"}
Sep 14 01:35:28.926018 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:28.925968    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jxz4h" podStartSLOduration=251.829604755 podStartE2EDuration="4m13.925950855s" totalImagesPullingTime="2.0963461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:31:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:28.924223634 +0000 UTC m=+288.404674456" watchObservedRunningTime="2026-09-14 01:35:28.925950855 +0000 UTC m=+288.406401676"
Sep 14 01:35:29.914882 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:29.914856    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:35:32.950598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:32.950559    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"]
Sep 14 01:35:32.953665 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:32.953648    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"
Sep 14 01:35:32.955529 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:32.955505    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 14 01:35:32.955640 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:32.955558    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 14 01:35:32.956119 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:32.956104    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-pbglt\""
Sep 14 01:35:32.956207 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:32.956182    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 14 01:35:32.962136 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:32.962112    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"]
Sep 14 01:35:33.040796 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:33.040758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e613f4c-b402-4660-b0b9-6aab3a126440-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-zb45z\" (UID: \"8e613f4c-b402-4660-b0b9-6aab3a126440\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"
Sep 14 01:35:33.040796 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:33.040806    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e613f4c-b402-4660-b0b9-6aab3a126440-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-zb45z\" (UID: \"8e613f4c-b402-4660-b0b9-6aab3a126440\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"
Sep 14 01:35:33.041057 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:33.040871    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e613f4c-b402-4660-b0b9-6aab3a126440-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-zb45z\" (UID: \"8e613f4c-b402-4660-b0b9-6aab3a126440\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"
Sep 14 01:35:33.041057 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:33.040942    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pmdt\" (UniqueName: \"kubernetes.io/projected/8e613f4c-b402-4660-b0b9-6aab3a126440-kube-api-access-5pmdt\") pod \"prometheus-operator-68bcd7f987-zb45z\" (UID: \"8e613f4c-b402-4660-b0b9-6aab3a126440\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"
Sep 14 01:35:33.141738 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:33.141703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pmdt\" (UniqueName: \"kubernetes.io/projected/8e613f4c-b402-4660-b0b9-6aab3a126440-kube-api-access-5pmdt\") pod \"prometheus-operator-68bcd7f987-zb45z\" (UID: \"8e613f4c-b402-4660-b0b9-6aab3a126440\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"
Sep 14 01:35:33.141738 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:33.141745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e613f4c-b402-4660-b0b9-6aab3a126440-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-zb45z\" (UID: \"8e613f4c-b402-4660-b0b9-6aab3a126440\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"
Sep 14 01:35:33.142039 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:33.141765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e613f4c-b402-4660-b0b9-6aab3a126440-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-zb45z\" (UID: \"8e613f4c-b402-4660-b0b9-6aab3a126440\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"
Sep 14 01:35:33.142039 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:33.141808    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e613f4c-b402-4660-b0b9-6aab3a126440-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-zb45z\" (UID: \"8e613f4c-b402-4660-b0b9-6aab3a126440\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"
Sep 14 01:35:33.142647 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:33.142624    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e613f4c-b402-4660-b0b9-6aab3a126440-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-zb45z\" (UID: \"8e613f4c-b402-4660-b0b9-6aab3a126440\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"
Sep 14 01:35:33.144418 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:33.144386    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e613f4c-b402-4660-b0b9-6aab3a126440-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-zb45z\" (UID: \"8e613f4c-b402-4660-b0b9-6aab3a126440\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"
Sep 14 01:35:33.144418 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:33.144385    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e613f4c-b402-4660-b0b9-6aab3a126440-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-zb45z\" (UID: \"8e613f4c-b402-4660-b0b9-6aab3a126440\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"
Sep 14 01:35:33.150961 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:33.150941    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pmdt\" (UniqueName: \"kubernetes.io/projected/8e613f4c-b402-4660-b0b9-6aab3a126440-kube-api-access-5pmdt\") pod \"prometheus-operator-68bcd7f987-zb45z\" (UID: \"8e613f4c-b402-4660-b0b9-6aab3a126440\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"
Sep 14 01:35:33.262959 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:33.262907    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"
Sep 14 01:35:33.530391 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:33.530311    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-zb45z"]
Sep 14 01:35:33.533506 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:33.533474    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e613f4c_b402_4660_b0b9_6aab3a126440.slice/crio-663b5ef58e4f32d4253a83ebe0c9bf4fa138c246e62cfe5712c7665bee6222a5 WatchSource:0}: Error finding container 663b5ef58e4f32d4253a83ebe0c9bf4fa138c246e62cfe5712c7665bee6222a5: Status 404 returned error can't find the container with id 663b5ef58e4f32d4253a83ebe0c9bf4fa138c246e62cfe5712c7665bee6222a5
Sep 14 01:35:33.929390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:33.929301    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z" event={"ID":"8e613f4c-b402-4660-b0b9-6aab3a126440","Type":"ContainerStarted","Data":"663b5ef58e4f32d4253a83ebe0c9bf4fa138c246e62cfe5712c7665bee6222a5"}
Sep 14 01:35:34.050407 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:34.050364    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fv6nk" podUID="ff2b27ad-c198-489c-9bb8-befeea447a64"
Sep 14 01:35:34.935106 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:34.935073    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z" event={"ID":"8e613f4c-b402-4660-b0b9-6aab3a126440","Type":"ContainerStarted","Data":"74fd187674ab05586d39bc758e12ea1bec58ccda55c3214359334f8e2165f3af"}
Sep 14 01:35:34.935265 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:34.935128    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z" event={"ID":"8e613f4c-b402-4660-b0b9-6aab3a126440","Type":"ContainerStarted","Data":"54cd49af4117b6ba08d0b4eee243f762885d1e0379977f9d6d2cbb02badb21a3"}
Sep 14 01:35:34.952197 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:34.952095    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-zb45z" podStartSLOduration=1.7913946570000001 podStartE2EDuration="2.952077418s" totalImagesPullingTime="1.160682761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:35:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:34.950244333 +0000 UTC m=+294.430695157" watchObservedRunningTime="2026-09-14 01:35:34.952077418 +0000 UTC m=+294.432528255"
Sep 14 01:35:35.605630 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.605590    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f5b57db46-888rr"]
Sep 14 01:35:35.608800 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.608771    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.610897 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.610875    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bmmfp\""
Sep 14 01:35:35.611034 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.610881    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 14 01:35:35.611034 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.610933    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 14 01:35:35.611517 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.611503    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 14 01:35:35.611625 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.611610    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 14 01:35:35.611681 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.611636    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 14 01:35:35.611681 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.611639    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 14 01:35:35.611767 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.611724    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 14 01:35:35.618613 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.618585    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f5b57db46-888rr"]
Sep 14 01:35:35.662944 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.662884    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-oauth-config\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.662944 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.662947    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-oauth-serving-cert\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.663156 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.662999    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-serving-cert\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.663156 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.663032    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjz4q\" (UniqueName: \"kubernetes.io/projected/8e585bb3-c6ca-49b9-a237-9a2971a73020-kube-api-access-jjz4q\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.663156 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.663087    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-config\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.663156 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.663110    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-service-ca\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.763997 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.763958    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-oauth-config\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.763997 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.763993    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-oauth-serving-cert\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.764234 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.764023    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-serving-cert\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.764234 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.764057    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjz4q\" (UniqueName: \"kubernetes.io/projected/8e585bb3-c6ca-49b9-a237-9a2971a73020-kube-api-access-jjz4q\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.764234 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.764134    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-config\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.764234 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.764162    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-service-ca\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.764815 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.764784    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-oauth-serving-cert\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.765001 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.764977    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-service-ca\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.765077 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.764982    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-config\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.766523 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.766500    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-serving-cert\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.766610 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.766586    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-oauth-config\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.771237 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.771220    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjz4q\" (UniqueName: \"kubernetes.io/projected/8e585bb3-c6ca-49b9-a237-9a2971a73020-kube-api-access-jjz4q\") pod \"console-7f5b57db46-888rr\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") " pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:35.918978 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:35.918860    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:36.191631 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:36.191608    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f5b57db46-888rr"]
Sep 14 01:35:36.193938 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:36.193897    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e585bb3_c6ca_49b9_a237_9a2971a73020.slice/crio-376bc0739e58e50b547c54cb7ba4e426cd1bc54ed961a22f3e90f83ffa4a21a8 WatchSource:0}: Error finding container 376bc0739e58e50b547c54cb7ba4e426cd1bc54ed961a22f3e90f83ffa4a21a8: Status 404 returned error can't find the container with id 376bc0739e58e50b547c54cb7ba4e426cd1bc54ed961a22f3e90f83ffa4a21a8
Sep 14 01:35:36.949381 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:36.949343    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5b57db46-888rr" event={"ID":"8e585bb3-c6ca-49b9-a237-9a2971a73020","Type":"ContainerStarted","Data":"376bc0739e58e50b547c54cb7ba4e426cd1bc54ed961a22f3e90f83ffa4a21a8"}
Sep 14 01:35:37.296790 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.296458    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"]
Sep 14 01:35:37.300394 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.300359    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"
Sep 14 01:35:37.302811 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.302623    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-s5tcw\""
Sep 14 01:35:37.303153 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.303038    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 14 01:35:37.303153 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.303059    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 14 01:35:37.309739 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.309670    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"]
Sep 14 01:35:37.325280 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.325242    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lbzvd"]
Sep 14 01:35:37.329146 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.329121    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.331642 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.331386    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 14 01:35:37.331642 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.331430    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 14 01:35:37.331642 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.331635    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wvnzr\""
Sep 14 01:35:37.332484 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.331693    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 14 01:35:37.376342 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.376247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-accelerators-collector-config\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.376342 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.376294    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2ff7849-fec9-4c2f-aef9-74830987667a-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-cpq8r\" (UID: \"b2ff7849-fec9-4c2f-aef9-74830987667a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"
Sep 14 01:35:37.376598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.376413    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-tls\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.376598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.376471    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-wtmp\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.376598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.376505    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b4mm\" (UniqueName: \"kubernetes.io/projected/b2ff7849-fec9-4c2f-aef9-74830987667a-kube-api-access-4b4mm\") pod \"openshift-state-metrics-67dcf8c979-cpq8r\" (UID: \"b2ff7849-fec9-4c2f-aef9-74830987667a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"
Sep 14 01:35:37.376598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.376534    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-textfile\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.376598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.376567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b2ff7849-fec9-4c2f-aef9-74830987667a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-cpq8r\" (UID: \"b2ff7849-fec9-4c2f-aef9-74830987667a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"
Sep 14 01:35:37.376856 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.376598    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2ff7849-fec9-4c2f-aef9-74830987667a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-cpq8r\" (UID: \"b2ff7849-fec9-4c2f-aef9-74830987667a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"
Sep 14 01:35:37.376856 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.376648    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/de2f4016-4c8f-40c7-ab01-f65c559befde-root\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.376856 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.376686    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de2f4016-4c8f-40c7-ab01-f65c559befde-metrics-client-ca\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.376856 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.376713    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de2f4016-4c8f-40c7-ab01-f65c559befde-sys\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.376856 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.376738    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.376856 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.376794    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn4dq\" (UniqueName: \"kubernetes.io/projected/de2f4016-4c8f-40c7-ab01-f65c559befde-kube-api-access-mn4dq\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.478202 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.478152    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de2f4016-4c8f-40c7-ab01-f65c559befde-metrics-client-ca\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.478202 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.478205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de2f4016-4c8f-40c7-ab01-f65c559befde-sys\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.478453 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.478227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.478453 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.478257    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn4dq\" (UniqueName: \"kubernetes.io/projected/de2f4016-4c8f-40c7-ab01-f65c559befde-kube-api-access-mn4dq\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.478453 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.478295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-accelerators-collector-config\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.478453 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.478324    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2ff7849-fec9-4c2f-aef9-74830987667a-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-cpq8r\" (UID: \"b2ff7849-fec9-4c2f-aef9-74830987667a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"
Sep 14 01:35:37.478453 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.478386    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-tls\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.478453 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.478430    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-wtmp\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.478748 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.478459    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4mm\" (UniqueName: \"kubernetes.io/projected/b2ff7849-fec9-4c2f-aef9-74830987667a-kube-api-access-4b4mm\") pod \"openshift-state-metrics-67dcf8c979-cpq8r\" (UID: \"b2ff7849-fec9-4c2f-aef9-74830987667a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"
Sep 14 01:35:37.478748 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.478488    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-textfile\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.478748 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.478515    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b2ff7849-fec9-4c2f-aef9-74830987667a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-cpq8r\" (UID: \"b2ff7849-fec9-4c2f-aef9-74830987667a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"
Sep 14 01:35:37.478748 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.478534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2ff7849-fec9-4c2f-aef9-74830987667a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-cpq8r\" (UID: \"b2ff7849-fec9-4c2f-aef9-74830987667a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"
Sep 14 01:35:37.478748 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.478549    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/de2f4016-4c8f-40c7-ab01-f65c559befde-root\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.478748 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.478629    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/de2f4016-4c8f-40c7-ab01-f65c559befde-root\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.479069 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:37.478966    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 14 01:35:37.479069 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:37.479036    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-tls podName:de2f4016-4c8f-40c7-ab01-f65c559befde nodeName:}" failed. No retries permitted until 2026-09-14 01:35:37.979016118 +0000 UTC m=+297.459466936 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-tls") pod "node-exporter-lbzvd" (UID: "de2f4016-4c8f-40c7-ab01-f65c559befde") : secret "node-exporter-tls" not found
Sep 14 01:35:37.479224 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.479201    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-textfile\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.479313 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.479290    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de2f4016-4c8f-40c7-ab01-f65c559befde-metrics-client-ca\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.479365 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.479353    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-wtmp\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.479656 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.479634    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-accelerators-collector-config\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.479710 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.479304    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de2f4016-4c8f-40c7-ab01-f65c559befde-sys\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.480066 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.480030    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2ff7849-fec9-4c2f-aef9-74830987667a-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-cpq8r\" (UID: \"b2ff7849-fec9-4c2f-aef9-74830987667a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"
Sep 14 01:35:37.482394 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.482362    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.483461 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.483260    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2ff7849-fec9-4c2f-aef9-74830987667a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-cpq8r\" (UID: \"b2ff7849-fec9-4c2f-aef9-74830987667a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"
Sep 14 01:35:37.484454 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.484399    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b2ff7849-fec9-4c2f-aef9-74830987667a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-cpq8r\" (UID: \"b2ff7849-fec9-4c2f-aef9-74830987667a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"
Sep 14 01:35:37.487125 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.487083    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn4dq\" (UniqueName: \"kubernetes.io/projected/de2f4016-4c8f-40c7-ab01-f65c559befde-kube-api-access-mn4dq\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.488141 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.488093    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b4mm\" (UniqueName: \"kubernetes.io/projected/b2ff7849-fec9-4c2f-aef9-74830987667a-kube-api-access-4b4mm\") pod \"openshift-state-metrics-67dcf8c979-cpq8r\" (UID: \"b2ff7849-fec9-4c2f-aef9-74830987667a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"
Sep 14 01:35:37.616047 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.615959    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"
Sep 14 01:35:37.903148 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.903061    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r"]
Sep 14 01:35:37.906590 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:37.906514    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2ff7849_fec9_4c2f_aef9_74830987667a.slice/crio-c581cc059b2829ca803c8426412aeda828063ebcc11e07f3c0d8a6f7859f7daf WatchSource:0}: Error finding container c581cc059b2829ca803c8426412aeda828063ebcc11e07f3c0d8a6f7859f7daf: Status 404 returned error can't find the container with id c581cc059b2829ca803c8426412aeda828063ebcc11e07f3c0d8a6f7859f7daf
Sep 14 01:35:37.955468 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.955436    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r" event={"ID":"b2ff7849-fec9-4c2f-aef9-74830987667a","Type":"ContainerStarted","Data":"c581cc059b2829ca803c8426412aeda828063ebcc11e07f3c0d8a6f7859f7daf"}
Sep 14 01:35:37.983022 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.982875    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-tls\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:37.985386 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:37.985362    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/de2f4016-4c8f-40c7-ab01-f65c559befde-node-exporter-tls\") pod \"node-exporter-lbzvd\" (UID: \"de2f4016-4c8f-40c7-ab01-f65c559befde\") " pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:38.241744 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:38.241639    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lbzvd"
Sep 14 01:35:38.253636 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:38.253602    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde2f4016_4c8f_40c7_ab01_f65c559befde.slice/crio-b965059502b515063d99f79d3559784508e3755b4f039d684bf4b9a882344a2c WatchSource:0}: Error finding container b965059502b515063d99f79d3559784508e3755b4f039d684bf4b9a882344a2c: Status 404 returned error can't find the container with id b965059502b515063d99f79d3559784508e3755b4f039d684bf4b9a882344a2c
Sep 14 01:35:38.962299 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:38.962227    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r" event={"ID":"b2ff7849-fec9-4c2f-aef9-74830987667a","Type":"ContainerStarted","Data":"80c0a78041a72811202e7c030e0ac4228cba5f23e8505c00f4693e4491c77486"}
Sep 14 01:35:38.962758 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:38.962310    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r" event={"ID":"b2ff7849-fec9-4c2f-aef9-74830987667a","Type":"ContainerStarted","Data":"dc1930557d250116549f8a728c44dcff8e42cf3e6389fa861e1631b3321e7b79"}
Sep 14 01:35:38.963637 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:38.963606    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lbzvd" event={"ID":"de2f4016-4c8f-40c7-ab01-f65c559befde","Type":"ContainerStarted","Data":"b965059502b515063d99f79d3559784508e3755b4f039d684bf4b9a882344a2c"}
Sep 14 01:35:39.269945 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.269839    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-cb589b864-gk2vw"]
Sep 14 01:35:39.278804 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.278768    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.280942 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.280897    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 14 01:35:39.281097 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.280901    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 14 01:35:39.281097 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.281008    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 14 01:35:39.281198 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.281179    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q574s\""
Sep 14 01:35:39.281252 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.281229    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 14 01:35:39.281304 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.281270    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ah8df6d5kmerv\""
Sep 14 01:35:39.281357 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.281312    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 14 01:35:39.284681 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.284660    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-cb589b864-gk2vw"]
Sep 14 01:35:39.397575 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.397534    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.397575 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.397577    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-grpc-tls\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.397815 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.397625    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.397815 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.397672    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-thanos-querier-tls\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.397815 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.397731    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.397815 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.397774    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzxz2\" (UniqueName: \"kubernetes.io/projected/9ee86a74-4f23-4850-94cf-0af95a3a5361-kube-api-access-zzxz2\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.397815 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.397812    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.398054 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.397838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ee86a74-4f23-4850-94cf-0af95a3a5361-metrics-client-ca\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.498589 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.498557    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.498736 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.498597    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-thanos-querier-tls\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.498736 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.498631    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.498736 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.498669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzxz2\" (UniqueName: \"kubernetes.io/projected/9ee86a74-4f23-4850-94cf-0af95a3a5361-kube-api-access-zzxz2\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.498736 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.498700    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.498736 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.498722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ee86a74-4f23-4850-94cf-0af95a3a5361-metrics-client-ca\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.499010 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.498904    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.499010 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.498978    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-grpc-tls\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.500785 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.499868    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ee86a74-4f23-4850-94cf-0af95a3a5361-metrics-client-ca\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.501785 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.501764    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-grpc-tls\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.501874 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.501780    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.501874 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.501852    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-thanos-querier-tls\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.502258 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.502070    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.502258 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.502125    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.502691 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.502670    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ee86a74-4f23-4850-94cf-0af95a3a5361-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.506574 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.506550    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzxz2\" (UniqueName: \"kubernetes.io/projected/9ee86a74-4f23-4850-94cf-0af95a3a5361-kube-api-access-zzxz2\") pod \"thanos-querier-cb589b864-gk2vw\" (UID: \"9ee86a74-4f23-4850-94cf-0af95a3a5361\") " pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.590682 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.590586    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:39.877095 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.877068    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-cb589b864-gk2vw"]
Sep 14 01:35:39.921019 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.920987    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jxz4h"
Sep 14 01:35:39.970578 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.970538    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5b57db46-888rr" event={"ID":"8e585bb3-c6ca-49b9-a237-9a2971a73020","Type":"ContainerStarted","Data":"d99c33f6731f8017de7b6dea733f343cd4cbd599209114b71293d7774eb083e6"}
Sep 14 01:35:39.984953 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:39.984872    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f5b57db46-888rr" podStartSLOduration=1.736680784 podStartE2EDuration="4.984850933s" totalImagesPullingTime="3.248170149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:35:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:39.984848888 +0000 UTC m=+299.465299713" watchObservedRunningTime="2026-09-14 01:35:39.984850933 +0000 UTC m=+299.465301757"
Sep 14 01:35:39.989284 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:39.989252    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ee86a74_4f23_4850_94cf_0af95a3a5361.slice/crio-ed07f06198c2afb9a3163dcff0a0af52fecb19f809457ffea777d5f238471b2e WatchSource:0}: Error finding container ed07f06198c2afb9a3163dcff0a0af52fecb19f809457ffea777d5f238471b2e: Status 404 returned error can't find the container with id ed07f06198c2afb9a3163dcff0a0af52fecb19f809457ffea777d5f238471b2e
Sep 14 01:35:40.970632 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:40.970472    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 01:35:40.970632 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:40.970600    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 01:35:40.977480 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:40.977442    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r" event={"ID":"b2ff7849-fec9-4c2f-aef9-74830987667a","Type":"ContainerStarted","Data":"cd7d53e3471fdcc5c14fc00cd3252c39afbaaabcb9661c7402560cd13f2b533c"}
Sep 14 01:35:40.979757 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:40.979725    2581 generic.go:356] "Generic (PLEG): container finished" podID="de2f4016-4c8f-40c7-ab01-f65c559befde" containerID="f07fd95b16840ac1d22f55120ef292b8ed1ba7174dc4df6e68f28c34ba70a20a" exitCode=0
Sep 14 01:35:40.979922 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:40.979845    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lbzvd" event={"ID":"de2f4016-4c8f-40c7-ab01-f65c559befde","Type":"ContainerDied","Data":"f07fd95b16840ac1d22f55120ef292b8ed1ba7174dc4df6e68f28c34ba70a20a"}
Sep 14 01:35:40.981108 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:40.981087    2581 kubelet.go:1648] "Image garbage collection succeeded"
Sep 14 01:35:40.981869 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:40.981843    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw" event={"ID":"9ee86a74-4f23-4850-94cf-0af95a3a5361","Type":"ContainerStarted","Data":"ed07f06198c2afb9a3163dcff0a0af52fecb19f809457ffea777d5f238471b2e"}
Sep 14 01:35:40.992686 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:40.992632    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-cpq8r" podStartSLOduration=1.777569933 podStartE2EDuration="3.992614808s" totalImagesPullingTime="2.215044875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:40.991732765 +0000 UTC m=+300.472183587" watchObservedRunningTime="2026-09-14 01:35:40.992614808 +0000 UTC m=+300.473065629"
Sep 14 01:35:41.988241 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:41.988206    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lbzvd" event={"ID":"de2f4016-4c8f-40c7-ab01-f65c559befde","Type":"ContainerStarted","Data":"18923e50330b0d510cc4ab8bd61d58b3cbf426d940d53adcb9e2998e19c3adc8"}
Sep 14 01:35:41.988630 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:41.988293    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lbzvd" event={"ID":"de2f4016-4c8f-40c7-ab01-f65c559befde","Type":"ContainerStarted","Data":"a895da800a446ca1e34e93f83a491eb262b668db5dc2b3880a74c5ee44faa3d9"}
Sep 14 01:35:42.012222 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.012166    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lbzvd" podStartSLOduration=2.942638625 podStartE2EDuration="5.012143158s" totalImagesPullingTime="1.999934848s" totalInitContainerRuntime="69.569685ms" isStatefulPod=true podCreationTimestamp="2026-09-14 01:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:42.010356336 +0000 UTC m=+301.490807158" watchObservedRunningTime="2026-09-14 01:35:42.012143158 +0000 UTC m=+301.492593979"
Sep 14 01:35:42.072156 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.072127    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp"]
Sep 14 01:35:42.076452 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.076431    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp"
Sep 14 01:35:42.078509 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.078488    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 14 01:35:42.085233 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.085213    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-vvndn\""
Sep 14 01:35:42.093178 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.093153    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp"]
Sep 14 01:35:42.126433 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.126396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-gdtkp\" (UID: \"3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp"
Sep 14 01:35:42.126594 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.126484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qthv\" (UniqueName: \"kubernetes.io/projected/3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1-kube-api-access-5qthv\") pod \"monitoring-plugin-5d85778b9d-gdtkp\" (UID: \"3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp"
Sep 14 01:35:42.227977 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.227854    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-gdtkp\" (UID: \"3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp"
Sep 14 01:35:42.227977 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.227896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qthv\" (UniqueName: \"kubernetes.io/projected/3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1-kube-api-access-5qthv\") pod \"monitoring-plugin-5d85778b9d-gdtkp\" (UID: \"3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp"
Sep 14 01:35:42.228187 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:42.228037    2581 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 14 01:35:42.228187 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:35:42.228107    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1-monitoring-plugin-cert podName:3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1 nodeName:}" failed. No retries permitted until 2026-09-14 01:35:42.72808944 +0000 UTC m=+302.208540239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-gdtkp" (UID: "3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1") : secret "monitoring-plugin-cert" not found
Sep 14 01:35:42.235369 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.235339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qthv\" (UniqueName: \"kubernetes.io/projected/3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1-kube-api-access-5qthv\") pod \"monitoring-plugin-5d85778b9d-gdtkp\" (UID: \"3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp"
Sep 14 01:35:42.732524 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.732489    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-gdtkp\" (UID: \"3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp"
Sep 14 01:35:42.735084 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.735059    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-gdtkp\" (UID: \"3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp"
Sep 14 01:35:42.995450 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.995355    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw" event={"ID":"9ee86a74-4f23-4850-94cf-0af95a3a5361","Type":"ContainerStarted","Data":"41b45ea8e5dfb5d3da20ef980b260fb8c487a72cdb5f0168627aca1b4f76f4aa"}
Sep 14 01:35:42.995857 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.995456    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw" event={"ID":"9ee86a74-4f23-4850-94cf-0af95a3a5361","Type":"ContainerStarted","Data":"0e8384bc7e8dde03ebbb984f2e6b75df148219e6f7307ac86b1613631dafd48b"}
Sep 14 01:35:42.995857 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.995500    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw" event={"ID":"9ee86a74-4f23-4850-94cf-0af95a3a5361","Type":"ContainerStarted","Data":"a9c649484d8efb640bbe1031c2429d12267be61eeb4225050c3a67ed33170d1e"}
Sep 14 01:35:42.996873 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:42.996840    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp"
Sep 14 01:35:43.270048 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:43.270014    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp"]
Sep 14 01:35:43.273137 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:43.273107    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e8b1fdd_1050_4b1e_8de5_11b9c4fd68a1.slice/crio-ea3abbfbc7b95b455f5f7b5a90e169cb8f104800e5b15dd06d2667cd8a289221 WatchSource:0}: Error finding container ea3abbfbc7b95b455f5f7b5a90e169cb8f104800e5b15dd06d2667cd8a289221: Status 404 returned error can't find the container with id ea3abbfbc7b95b455f5f7b5a90e169cb8f104800e5b15dd06d2667cd8a289221
Sep 14 01:35:43.280601 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:43.280574    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 01:35:44.001105 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:44.001062    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp" event={"ID":"3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1","Type":"ContainerStarted","Data":"ea3abbfbc7b95b455f5f7b5a90e169cb8f104800e5b15dd06d2667cd8a289221"}
Sep 14 01:35:44.004479 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:44.004450    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw" event={"ID":"9ee86a74-4f23-4850-94cf-0af95a3a5361","Type":"ContainerStarted","Data":"c1047229c0992285af7b24d8146da237f896822ec75e7ee317759f550d83a3b5"}
Sep 14 01:35:44.004630 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:44.004502    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw" event={"ID":"9ee86a74-4f23-4850-94cf-0af95a3a5361","Type":"ContainerStarted","Data":"001dc89553ea26e880b9f18a865d97aea2523299714cf952677541a38b995d99"}
Sep 14 01:35:44.004630 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:44.004529    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw" event={"ID":"9ee86a74-4f23-4850-94cf-0af95a3a5361","Type":"ContainerStarted","Data":"cbb8f0ca5a12c6a3e8dfd4205e349312edc98fd4f59626f1ebb5c174356ecc1a"}
Sep 14 01:35:44.004750 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:44.004653    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:44.025089 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:44.025039    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw" podStartSLOduration=2.049669516 podStartE2EDuration="5.025024704s" totalImagesPullingTime="2.975355188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:35:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:44.022775667 +0000 UTC m=+303.503226513" watchObservedRunningTime="2026-09-14 01:35:44.025024704 +0000 UTC m=+303.505475560"
Sep 14 01:35:45.009980 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:45.009946    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp" event={"ID":"3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1","Type":"ContainerStarted","Data":"914394c079ed54866a00ebe21fc0a506207be565f4f7a308e9be3df7d2b4b0e2"}
Sep 14 01:35:45.010411 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:45.010308    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp"
Sep 14 01:35:45.014938 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:45.014903    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp"
Sep 14 01:35:45.024484 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:45.024440    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-gdtkp" podStartSLOduration=1.7488196999999999 podStartE2EDuration="3.02442528s" totalImagesPullingTime="1.27560558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:35:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:45.023752309 +0000 UTC m=+304.504203129" watchObservedRunningTime="2026-09-14 01:35:45.02442528 +0000 UTC m=+304.504876101"
Sep 14 01:35:45.471652 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:45.471612    2581 patch_prober.go:28] interesting pod/image-registry-56cb6897db-dvtww container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 14 01:35:45.471812 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:45.471665    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-56cb6897db-dvtww" podUID="526bc8e9-328b-4c29-912e-1e092a8b97fa" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 01:35:45.919074 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:45.918991    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:45.919074 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:45.919055    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:45.923567 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:45.923543    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:46.018333 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.018308    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:35:46.201376 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.201284    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78785f98d9-z8cfb"]
Sep 14 01:35:46.204761 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.204738    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.210474 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.210450    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 14 01:35:46.214483 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.214454    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78785f98d9-z8cfb"]
Sep 14 01:35:46.267789 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.267753    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-oauth-serving-cert\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.267977 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.267793    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/202e1539-3daf-4627-b6c4-852744f3ddcb-console-serving-cert\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.267977 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.267863    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-console-config\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.267977 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.267963    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/202e1539-3daf-4627-b6c4-852744f3ddcb-console-oauth-config\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.268087 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.268053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfh24\" (UniqueName: \"kubernetes.io/projected/202e1539-3daf-4627-b6c4-852744f3ddcb-kube-api-access-lfh24\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.268087 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.268069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-trusted-ca-bundle\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.268153 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.268118    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-service-ca\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.369176 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.369133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfh24\" (UniqueName: \"kubernetes.io/projected/202e1539-3daf-4627-b6c4-852744f3ddcb-kube-api-access-lfh24\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.369176 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.369175    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-trusted-ca-bundle\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.369433 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.369230    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-service-ca\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.369433 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.369259    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-oauth-serving-cert\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.369433 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.369375    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/202e1539-3daf-4627-b6c4-852744f3ddcb-console-serving-cert\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.369635 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.369612    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-console-config\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.369704 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.369689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/202e1539-3daf-4627-b6c4-852744f3ddcb-console-oauth-config\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.370152 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.370126    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-service-ca\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.370282 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.370175    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-oauth-serving-cert\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.370341 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.370305    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-console-config\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.370410 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.370392    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-trusted-ca-bundle\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.371991 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.371965    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/202e1539-3daf-4627-b6c4-852744f3ddcb-console-serving-cert\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.372085 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.372040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/202e1539-3daf-4627-b6c4-852744f3ddcb-console-oauth-config\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.376037 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.376016    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfh24\" (UniqueName: \"kubernetes.io/projected/202e1539-3daf-4627-b6c4-852744f3ddcb-kube-api-access-lfh24\") pod \"console-78785f98d9-z8cfb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") " pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.515408 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.515356    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:46.787132 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.787108    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78785f98d9-z8cfb"]
Sep 14 01:35:46.789800 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:46.789772    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod202e1539_3daf_4627_b6c4_852744f3ddcb.slice/crio-fb64520a0cbfff47b991cf2c2b2da3e7417b0cb777b66649a8730e8490b33cab WatchSource:0}: Error finding container fb64520a0cbfff47b991cf2c2b2da3e7417b0cb777b66649a8730e8490b33cab: Status 404 returned error can't find the container with id fb64520a0cbfff47b991cf2c2b2da3e7417b0cb777b66649a8730e8490b33cab
Sep 14 01:35:46.904408 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:46.904379    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:35:47.020998 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:47.020951    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78785f98d9-z8cfb" event={"ID":"202e1539-3daf-4627-b6c4-852744f3ddcb","Type":"ContainerStarted","Data":"745ab49f2f6a5d91089617e0d0c2893ab8b4a36d63f70b43a3ff553776c81ba1"}
Sep 14 01:35:47.021398 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:47.021069    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78785f98d9-z8cfb" event={"ID":"202e1539-3daf-4627-b6c4-852744f3ddcb","Type":"ContainerStarted","Data":"fb64520a0cbfff47b991cf2c2b2da3e7417b0cb777b66649a8730e8490b33cab"}
Sep 14 01:35:47.043384 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:47.043284    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78785f98d9-z8cfb" podStartSLOduration=1.04326552 podStartE2EDuration="1.04326552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:35:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:47.042362063 +0000 UTC m=+306.522812885" watchObservedRunningTime="2026-09-14 01:35:47.04326552 +0000 UTC m=+306.523716340"
Sep 14 01:35:47.047585 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:47.047559    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:35:50.016138 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:50.016111    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-cb589b864-gk2vw"
Sep 14 01:35:52.109976 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:52.109940    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56cb6897db-dvtww"]
Sep 14 01:35:56.206857 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.206824    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78785f98d9-z8cfb"]
Sep 14 01:35:56.231549 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.231520    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-654bc7596-22qk9"]
Sep 14 01:35:56.237468 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.237443    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.243388 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.243359    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654bc7596-22qk9"]
Sep 14 01:35:56.358249 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.358209    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-trusted-ca-bundle\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.358249 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.358250    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-oauth-config\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.358452 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.358271    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-oauth-serving-cert\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.358452 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.358356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-serving-cert\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.358452 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.358389    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf9s7\" (UniqueName: \"kubernetes.io/projected/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-kube-api-access-zf9s7\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.358452 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.358448    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-service-ca\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.358578 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.358480    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-config\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.459638 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.459560    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-serving-cert\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.459638 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.459598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf9s7\" (UniqueName: \"kubernetes.io/projected/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-kube-api-access-zf9s7\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.459999 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.459788    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-service-ca\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.459999 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.459950    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-config\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.459999 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.459992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-trusted-ca-bundle\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.460209 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.460031    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-oauth-config\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.460209 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.460070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-oauth-serving-cert\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.460619 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.460589    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-service-ca\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.460758 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.460734    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-oauth-serving-cert\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.460849 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.460754    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-config\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.460849 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.460835    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-trusted-ca-bundle\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.462250 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.462205    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-serving-cert\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.462358 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.462261    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-oauth-config\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.466453 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.466432    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf9s7\" (UniqueName: \"kubernetes.io/projected/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-kube-api-access-zf9s7\") pod \"console-654bc7596-22qk9\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") " pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.515514 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.515478    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:35:56.547771 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.547733    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:35:56.821193 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:56.821162    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654bc7596-22qk9"]
Sep 14 01:35:56.823609 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:35:56.823579    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b3b5c6_d14e_43cd_81cb_2af80cd60acf.slice/crio-ab2cd61959f59940ea6003b4758193e104b57b9b13ddd2f0a588fbff2669a9e2 WatchSource:0}: Error finding container ab2cd61959f59940ea6003b4758193e104b57b9b13ddd2f0a588fbff2669a9e2: Status 404 returned error can't find the container with id ab2cd61959f59940ea6003b4758193e104b57b9b13ddd2f0a588fbff2669a9e2
Sep 14 01:35:57.061803 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:57.061768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654bc7596-22qk9" event={"ID":"81b3b5c6-d14e-43cd-81cb-2af80cd60acf","Type":"ContainerStarted","Data":"0f8f684387e011b7ed5a8a611b3a9572bad5d1d6d591806d03000a0e95b8b5c4"}
Sep 14 01:35:57.061987 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:57.061819    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654bc7596-22qk9" event={"ID":"81b3b5c6-d14e-43cd-81cb-2af80cd60acf","Type":"ContainerStarted","Data":"ab2cd61959f59940ea6003b4758193e104b57b9b13ddd2f0a588fbff2669a9e2"}
Sep 14 01:35:57.078950 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:35:57.078831    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-654bc7596-22qk9" podStartSLOduration=1.078812233 podStartE2EDuration="1.078812233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:35:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:35:57.077120322 +0000 UTC m=+316.557571134" watchObservedRunningTime="2026-09-14 01:35:57.078812233 +0000 UTC m=+316.559263054"
Sep 14 01:36:06.548036 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:06.547993    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:36:06.548588 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:06.548082    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:36:06.552949 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:06.552926    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:36:07.103165 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:07.103137    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:36:07.144185 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:07.144155    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f5b57db46-888rr"]
Sep 14 01:36:13.126269 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:13.126235    2581 generic.go:356] "Generic (PLEG): container finished" podID="624bd393-bc43-4a14-a0de-cf8ba209b0ed" containerID="3b997a86fb00fb4492b1c07206015750c02a7e53b42f7022f59b5d05cdde75f5" exitCode=0
Sep 14 01:36:13.126644 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:13.126308    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-crc4t" event={"ID":"624bd393-bc43-4a14-a0de-cf8ba209b0ed","Type":"ContainerDied","Data":"3b997a86fb00fb4492b1c07206015750c02a7e53b42f7022f59b5d05cdde75f5"}
Sep 14 01:36:13.126695 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:13.126659    2581 scope.go:117] "RemoveContainer" containerID="3b997a86fb00fb4492b1c07206015750c02a7e53b42f7022f59b5d05cdde75f5"
Sep 14 01:36:14.132546 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:14.132512    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-crc4t" event={"ID":"624bd393-bc43-4a14-a0de-cf8ba209b0ed","Type":"ContainerStarted","Data":"60585e2130f671d15ad6466500e31298ee5080ddcceacba0c3173a48743ca988"}
Sep 14 01:36:17.129230 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.129187    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-56cb6897db-dvtww" podUID="526bc8e9-328b-4c29-912e-1e092a8b97fa" containerName="registry" containerID="cri-o://1a790a673101e0acf089db196f967e32e7de8c1d278417abd7d0953cd13a3b38" gracePeriod=30
Sep 14 01:36:17.381534 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.381470    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:36:17.448095 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.448060    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7qnf\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-kube-api-access-h7qnf\") pod \"526bc8e9-328b-4c29-912e-1e092a8b97fa\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") "
Sep 14 01:36:17.448265 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.448100    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/526bc8e9-328b-4c29-912e-1e092a8b97fa-installation-pull-secrets\") pod \"526bc8e9-328b-4c29-912e-1e092a8b97fa\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") "
Sep 14 01:36:17.448265 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.448161    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/526bc8e9-328b-4c29-912e-1e092a8b97fa-image-registry-private-configuration\") pod \"526bc8e9-328b-4c29-912e-1e092a8b97fa\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") "
Sep 14 01:36:17.448265 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.448196    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls\") pod \"526bc8e9-328b-4c29-912e-1e092a8b97fa\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") "
Sep 14 01:36:17.448265 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.448213    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/526bc8e9-328b-4c29-912e-1e092a8b97fa-trusted-ca\") pod \"526bc8e9-328b-4c29-912e-1e092a8b97fa\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") "
Sep 14 01:36:17.448265 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.448235    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-certificates\") pod \"526bc8e9-328b-4c29-912e-1e092a8b97fa\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") "
Sep 14 01:36:17.448265 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.448264    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-bound-sa-token\") pod \"526bc8e9-328b-4c29-912e-1e092a8b97fa\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") "
Sep 14 01:36:17.448591 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.448303    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/526bc8e9-328b-4c29-912e-1e092a8b97fa-ca-trust-extracted\") pod \"526bc8e9-328b-4c29-912e-1e092a8b97fa\" (UID: \"526bc8e9-328b-4c29-912e-1e092a8b97fa\") "
Sep 14 01:36:17.448795 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.448762    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/526bc8e9-328b-4c29-912e-1e092a8b97fa-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "526bc8e9-328b-4c29-912e-1e092a8b97fa" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:36:17.449160 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.449121    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "526bc8e9-328b-4c29-912e-1e092a8b97fa" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:36:17.450842 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.450769    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/526bc8e9-328b-4c29-912e-1e092a8b97fa-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "526bc8e9-328b-4c29-912e-1e092a8b97fa" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:36:17.450968 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.450900    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "526bc8e9-328b-4c29-912e-1e092a8b97fa" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:36:17.450968 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.450952    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "526bc8e9-328b-4c29-912e-1e092a8b97fa" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:36:17.451084 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.451033    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/526bc8e9-328b-4c29-912e-1e092a8b97fa-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "526bc8e9-328b-4c29-912e-1e092a8b97fa" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:36:17.451136 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.451081    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-kube-api-access-h7qnf" (OuterVolumeSpecName: "kube-api-access-h7qnf") pod "526bc8e9-328b-4c29-912e-1e092a8b97fa" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa"). InnerVolumeSpecName "kube-api-access-h7qnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:36:17.457179 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.457154    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/526bc8e9-328b-4c29-912e-1e092a8b97fa-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "526bc8e9-328b-4c29-912e-1e092a8b97fa" (UID: "526bc8e9-328b-4c29-912e-1e092a8b97fa"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:36:17.549304 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.549266    2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:17.549304 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.549297    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/526bc8e9-328b-4c29-912e-1e092a8b97fa-trusted-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:17.549304 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.549306    2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/526bc8e9-328b-4c29-912e-1e092a8b97fa-registry-certificates\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:17.549534 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.549317    2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-bound-sa-token\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:17.549534 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.549327    2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/526bc8e9-328b-4c29-912e-1e092a8b97fa-ca-trust-extracted\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:17.549534 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.549336    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7qnf\" (UniqueName: \"kubernetes.io/projected/526bc8e9-328b-4c29-912e-1e092a8b97fa-kube-api-access-h7qnf\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:17.549534 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.549345    2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/526bc8e9-328b-4c29-912e-1e092a8b97fa-installation-pull-secrets\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:17.549534 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:17.549355    2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/526bc8e9-328b-4c29-912e-1e092a8b97fa-image-registry-private-configuration\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:18.152145 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:18.152109    2581 generic.go:356] "Generic (PLEG): container finished" podID="526bc8e9-328b-4c29-912e-1e092a8b97fa" containerID="1a790a673101e0acf089db196f967e32e7de8c1d278417abd7d0953cd13a3b38" exitCode=0
Sep 14 01:36:18.152660 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:18.152151    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56cb6897db-dvtww"
Sep 14 01:36:18.152660 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:18.152176    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56cb6897db-dvtww" event={"ID":"526bc8e9-328b-4c29-912e-1e092a8b97fa","Type":"ContainerDied","Data":"1a790a673101e0acf089db196f967e32e7de8c1d278417abd7d0953cd13a3b38"}
Sep 14 01:36:18.152660 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:18.152224    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56cb6897db-dvtww" event={"ID":"526bc8e9-328b-4c29-912e-1e092a8b97fa","Type":"ContainerDied","Data":"e1dc05826bebd71c66c6b453bc6347db08b321c9c3f15f783dc05d61840f16fb"}
Sep 14 01:36:18.152660 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:18.152252    2581 scope.go:117] "RemoveContainer" containerID="1a790a673101e0acf089db196f967e32e7de8c1d278417abd7d0953cd13a3b38"
Sep 14 01:36:18.161066 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:18.161045    2581 scope.go:117] "RemoveContainer" containerID="1a790a673101e0acf089db196f967e32e7de8c1d278417abd7d0953cd13a3b38"
Sep 14 01:36:18.161320 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:36:18.161302    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a790a673101e0acf089db196f967e32e7de8c1d278417abd7d0953cd13a3b38\": container with ID starting with 1a790a673101e0acf089db196f967e32e7de8c1d278417abd7d0953cd13a3b38 not found: ID does not exist" containerID="1a790a673101e0acf089db196f967e32e7de8c1d278417abd7d0953cd13a3b38"
Sep 14 01:36:18.161363 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:18.161328    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a790a673101e0acf089db196f967e32e7de8c1d278417abd7d0953cd13a3b38"} err="failed to get container status \"1a790a673101e0acf089db196f967e32e7de8c1d278417abd7d0953cd13a3b38\": rpc error: code = NotFound desc = could not find container \"1a790a673101e0acf089db196f967e32e7de8c1d278417abd7d0953cd13a3b38\": container with ID starting with 1a790a673101e0acf089db196f967e32e7de8c1d278417abd7d0953cd13a3b38 not found: ID does not exist"
Sep 14 01:36:18.171004 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:18.170979    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56cb6897db-dvtww"]
Sep 14 01:36:18.174657 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:18.174630    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-56cb6897db-dvtww"]
Sep 14 01:36:19.052600 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:19.052569    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="526bc8e9-328b-4c29-912e-1e092a8b97fa" path="/var/lib/kubelet/pods/526bc8e9-328b-4c29-912e-1e092a8b97fa/volumes"
Sep 14 01:36:21.226038 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.226003    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78785f98d9-z8cfb" podUID="202e1539-3daf-4627-b6c4-852744f3ddcb" containerName="console" containerID="cri-o://745ab49f2f6a5d91089617e0d0c2893ab8b4a36d63f70b43a3ff553776c81ba1" gracePeriod=15
Sep 14 01:36:21.477237 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.477172    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:36:21.584234 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.584200    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/202e1539-3daf-4627-b6c4-852744f3ddcb-console-serving-cert\") pod \"202e1539-3daf-4627-b6c4-852744f3ddcb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") "
Sep 14 01:36:21.584234 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.584242    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfh24\" (UniqueName: \"kubernetes.io/projected/202e1539-3daf-4627-b6c4-852744f3ddcb-kube-api-access-lfh24\") pod \"202e1539-3daf-4627-b6c4-852744f3ddcb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") "
Sep 14 01:36:21.584477 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.584272    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-service-ca\") pod \"202e1539-3daf-4627-b6c4-852744f3ddcb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") "
Sep 14 01:36:21.584477 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.584318    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-console-config\") pod \"202e1539-3daf-4627-b6c4-852744f3ddcb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") "
Sep 14 01:36:21.584477 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.584349    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-oauth-serving-cert\") pod \"202e1539-3daf-4627-b6c4-852744f3ddcb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") "
Sep 14 01:36:21.584477 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.584386    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/202e1539-3daf-4627-b6c4-852744f3ddcb-console-oauth-config\") pod \"202e1539-3daf-4627-b6c4-852744f3ddcb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") "
Sep 14 01:36:21.584477 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.584414    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-trusted-ca-bundle\") pod \"202e1539-3daf-4627-b6c4-852744f3ddcb\" (UID: \"202e1539-3daf-4627-b6c4-852744f3ddcb\") "
Sep 14 01:36:21.584747 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.584723    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-service-ca" (OuterVolumeSpecName: "service-ca") pod "202e1539-3daf-4627-b6c4-852744f3ddcb" (UID: "202e1539-3daf-4627-b6c4-852744f3ddcb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:36:21.584814 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.584756    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-console-config" (OuterVolumeSpecName: "console-config") pod "202e1539-3daf-4627-b6c4-852744f3ddcb" (UID: "202e1539-3daf-4627-b6c4-852744f3ddcb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:36:21.584900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.584876    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "202e1539-3daf-4627-b6c4-852744f3ddcb" (UID: "202e1539-3daf-4627-b6c4-852744f3ddcb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:36:21.585035 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.585014    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "202e1539-3daf-4627-b6c4-852744f3ddcb" (UID: "202e1539-3daf-4627-b6c4-852744f3ddcb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:36:21.586737 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.586715    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/202e1539-3daf-4627-b6c4-852744f3ddcb-kube-api-access-lfh24" (OuterVolumeSpecName: "kube-api-access-lfh24") pod "202e1539-3daf-4627-b6c4-852744f3ddcb" (UID: "202e1539-3daf-4627-b6c4-852744f3ddcb"). InnerVolumeSpecName "kube-api-access-lfh24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:36:21.587010 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.586986    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/202e1539-3daf-4627-b6c4-852744f3ddcb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "202e1539-3daf-4627-b6c4-852744f3ddcb" (UID: "202e1539-3daf-4627-b6c4-852744f3ddcb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:36:21.587010 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.587000    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/202e1539-3daf-4627-b6c4-852744f3ddcb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "202e1539-3daf-4627-b6c4-852744f3ddcb" (UID: "202e1539-3daf-4627-b6c4-852744f3ddcb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:36:21.685254 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.685215    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/202e1539-3daf-4627-b6c4-852744f3ddcb-console-serving-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:21.685254 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.685246    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfh24\" (UniqueName: \"kubernetes.io/projected/202e1539-3daf-4627-b6c4-852744f3ddcb-kube-api-access-lfh24\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:21.685254 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.685256    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:21.685254 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.685264    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-console-config\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:21.685509 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.685274    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-oauth-serving-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:21.685509 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.685282    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/202e1539-3daf-4627-b6c4-852744f3ddcb-console-oauth-config\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:21.685509 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:21.685290    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/202e1539-3daf-4627-b6c4-852744f3ddcb-trusted-ca-bundle\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:22.170626 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:22.170586    2581 generic.go:356] "Generic (PLEG): container finished" podID="202e1539-3daf-4627-b6c4-852744f3ddcb" containerID="745ab49f2f6a5d91089617e0d0c2893ab8b4a36d63f70b43a3ff553776c81ba1" exitCode=0
Sep 14 01:36:22.170796 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:22.170661    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78785f98d9-z8cfb"
Sep 14 01:36:22.170796 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:22.170674    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78785f98d9-z8cfb" event={"ID":"202e1539-3daf-4627-b6c4-852744f3ddcb","Type":"ContainerDied","Data":"745ab49f2f6a5d91089617e0d0c2893ab8b4a36d63f70b43a3ff553776c81ba1"}
Sep 14 01:36:22.170796 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:22.170728    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78785f98d9-z8cfb" event={"ID":"202e1539-3daf-4627-b6c4-852744f3ddcb","Type":"ContainerDied","Data":"fb64520a0cbfff47b991cf2c2b2da3e7417b0cb777b66649a8730e8490b33cab"}
Sep 14 01:36:22.170796 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:22.170749    2581 scope.go:117] "RemoveContainer" containerID="745ab49f2f6a5d91089617e0d0c2893ab8b4a36d63f70b43a3ff553776c81ba1"
Sep 14 01:36:22.179904 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:22.179886    2581 scope.go:117] "RemoveContainer" containerID="745ab49f2f6a5d91089617e0d0c2893ab8b4a36d63f70b43a3ff553776c81ba1"
Sep 14 01:36:22.180253 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:36:22.180235    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"745ab49f2f6a5d91089617e0d0c2893ab8b4a36d63f70b43a3ff553776c81ba1\": container with ID starting with 745ab49f2f6a5d91089617e0d0c2893ab8b4a36d63f70b43a3ff553776c81ba1 not found: ID does not exist" containerID="745ab49f2f6a5d91089617e0d0c2893ab8b4a36d63f70b43a3ff553776c81ba1"
Sep 14 01:36:22.180316 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:22.180267    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"745ab49f2f6a5d91089617e0d0c2893ab8b4a36d63f70b43a3ff553776c81ba1"} err="failed to get container status \"745ab49f2f6a5d91089617e0d0c2893ab8b4a36d63f70b43a3ff553776c81ba1\": rpc error: code = NotFound desc = could not find container \"745ab49f2f6a5d91089617e0d0c2893ab8b4a36d63f70b43a3ff553776c81ba1\": container with ID starting with 745ab49f2f6a5d91089617e0d0c2893ab8b4a36d63f70b43a3ff553776c81ba1 not found: ID does not exist"
Sep 14 01:36:22.190002 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:22.189970    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78785f98d9-z8cfb"]
Sep 14 01:36:22.195464 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:22.195443    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78785f98d9-z8cfb"]
Sep 14 01:36:23.052866 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:23.052824    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="202e1539-3daf-4627-b6c4-852744f3ddcb" path="/var/lib/kubelet/pods/202e1539-3daf-4627-b6c4-852744f3ddcb/volumes"
Sep 14 01:36:23.176242 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:23.176208    2581 generic.go:356] "Generic (PLEG): container finished" podID="066d2d8c-90ad-48fd-bd89-941f4c0e3c8c" containerID="79cb8a67b0c03ea6ecdf18502ca697acd5219e1b99cd1981e083e740b3a78128" exitCode=0
Sep 14 01:36:23.176422 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:23.176283    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98" event={"ID":"066d2d8c-90ad-48fd-bd89-941f4c0e3c8c","Type":"ContainerDied","Data":"79cb8a67b0c03ea6ecdf18502ca697acd5219e1b99cd1981e083e740b3a78128"}
Sep 14 01:36:23.176698 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:23.176682    2581 scope.go:117] "RemoveContainer" containerID="79cb8a67b0c03ea6ecdf18502ca697acd5219e1b99cd1981e083e740b3a78128"
Sep 14 01:36:24.184976 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:24.184942    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqt98" event={"ID":"066d2d8c-90ad-48fd-bd89-941f4c0e3c8c","Type":"ContainerStarted","Data":"8bb2203c2fe24c7b969355db2886bed21a5d99e8f2387c63f0dbdaaae96a43a8"}
Sep 14 01:36:32.170247 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.170199    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f5b57db46-888rr" podUID="8e585bb3-c6ca-49b9-a237-9a2971a73020" containerName="console" containerID="cri-o://d99c33f6731f8017de7b6dea733f343cd4cbd599209114b71293d7774eb083e6" gracePeriod=15
Sep 14 01:36:32.416542 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.416517    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:36:32.477465 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.477439    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-service-ca\") pod \"8e585bb3-c6ca-49b9-a237-9a2971a73020\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") "
Sep 14 01:36:32.477625 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.477477    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-oauth-serving-cert\") pod \"8e585bb3-c6ca-49b9-a237-9a2971a73020\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") "
Sep 14 01:36:32.477625 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.477509    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-serving-cert\") pod \"8e585bb3-c6ca-49b9-a237-9a2971a73020\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") "
Sep 14 01:36:32.477625 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.477591    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-config\") pod \"8e585bb3-c6ca-49b9-a237-9a2971a73020\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") "
Sep 14 01:36:32.477625 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.477618    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-oauth-config\") pod \"8e585bb3-c6ca-49b9-a237-9a2971a73020\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") "
Sep 14 01:36:32.477818 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.477651    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjz4q\" (UniqueName: \"kubernetes.io/projected/8e585bb3-c6ca-49b9-a237-9a2971a73020-kube-api-access-jjz4q\") pod \"8e585bb3-c6ca-49b9-a237-9a2971a73020\" (UID: \"8e585bb3-c6ca-49b9-a237-9a2971a73020\") "
Sep 14 01:36:32.478020 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.477979    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8e585bb3-c6ca-49b9-a237-9a2971a73020" (UID: "8e585bb3-c6ca-49b9-a237-9a2971a73020"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:36:32.478149 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.478054    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-service-ca" (OuterVolumeSpecName: "service-ca") pod "8e585bb3-c6ca-49b9-a237-9a2971a73020" (UID: "8e585bb3-c6ca-49b9-a237-9a2971a73020"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:36:32.478190 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.478144    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-config" (OuterVolumeSpecName: "console-config") pod "8e585bb3-c6ca-49b9-a237-9a2971a73020" (UID: "8e585bb3-c6ca-49b9-a237-9a2971a73020"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:36:32.479943 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.479889    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8e585bb3-c6ca-49b9-a237-9a2971a73020" (UID: "8e585bb3-c6ca-49b9-a237-9a2971a73020"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:36:32.479943 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.479892    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e585bb3-c6ca-49b9-a237-9a2971a73020-kube-api-access-jjz4q" (OuterVolumeSpecName: "kube-api-access-jjz4q") pod "8e585bb3-c6ca-49b9-a237-9a2971a73020" (UID: "8e585bb3-c6ca-49b9-a237-9a2971a73020"). InnerVolumeSpecName "kube-api-access-jjz4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:36:32.479943 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.479908    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8e585bb3-c6ca-49b9-a237-9a2971a73020" (UID: "8e585bb3-c6ca-49b9-a237-9a2971a73020"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:36:32.578632 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.578594    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-config\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:32.578632 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.578626    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-oauth-config\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:32.578632 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.578637    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjz4q\" (UniqueName: \"kubernetes.io/projected/8e585bb3-c6ca-49b9-a237-9a2971a73020-kube-api-access-jjz4q\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:32.578632 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.578646    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:32.578878 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.578654    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e585bb3-c6ca-49b9-a237-9a2971a73020-oauth-serving-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:32.578878 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:32.578663    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e585bb3-c6ca-49b9-a237-9a2971a73020-console-serving-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:36:33.224628 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:33.224556    2581 generic.go:356] "Generic (PLEG): container finished" podID="8e585bb3-c6ca-49b9-a237-9a2971a73020" containerID="d99c33f6731f8017de7b6dea733f343cd4cbd599209114b71293d7774eb083e6" exitCode=0
Sep 14 01:36:33.224628 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:33.224603    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5b57db46-888rr"
Sep 14 01:36:33.225056 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:33.224636    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5b57db46-888rr" event={"ID":"8e585bb3-c6ca-49b9-a237-9a2971a73020","Type":"ContainerDied","Data":"d99c33f6731f8017de7b6dea733f343cd4cbd599209114b71293d7774eb083e6"}
Sep 14 01:36:33.225056 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:33.224701    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5b57db46-888rr" event={"ID":"8e585bb3-c6ca-49b9-a237-9a2971a73020","Type":"ContainerDied","Data":"376bc0739e58e50b547c54cb7ba4e426cd1bc54ed961a22f3e90f83ffa4a21a8"}
Sep 14 01:36:33.225056 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:33.224730    2581 scope.go:117] "RemoveContainer" containerID="d99c33f6731f8017de7b6dea733f343cd4cbd599209114b71293d7774eb083e6"
Sep 14 01:36:33.233473 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:33.233456    2581 scope.go:117] "RemoveContainer" containerID="d99c33f6731f8017de7b6dea733f343cd4cbd599209114b71293d7774eb083e6"
Sep 14 01:36:33.233748 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:36:33.233731    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d99c33f6731f8017de7b6dea733f343cd4cbd599209114b71293d7774eb083e6\": container with ID starting with d99c33f6731f8017de7b6dea733f343cd4cbd599209114b71293d7774eb083e6 not found: ID does not exist" containerID="d99c33f6731f8017de7b6dea733f343cd4cbd599209114b71293d7774eb083e6"
Sep 14 01:36:33.233795 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:33.233757    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d99c33f6731f8017de7b6dea733f343cd4cbd599209114b71293d7774eb083e6"} err="failed to get container status \"d99c33f6731f8017de7b6dea733f343cd4cbd599209114b71293d7774eb083e6\": rpc error: code = NotFound desc = could not find container \"d99c33f6731f8017de7b6dea733f343cd4cbd599209114b71293d7774eb083e6\": container with ID starting with d99c33f6731f8017de7b6dea733f343cd4cbd599209114b71293d7774eb083e6 not found: ID does not exist"
Sep 14 01:36:33.239664 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:33.239643    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f5b57db46-888rr"]
Sep 14 01:36:33.243716 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:33.243695    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f5b57db46-888rr"]
Sep 14 01:36:35.052966 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:35.052934    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e585bb3-c6ca-49b9-a237-9a2971a73020" path="/var/lib/kubelet/pods/8e585bb3-c6ca-49b9-a237-9a2971a73020/volumes"
Sep 14 01:36:55.081177 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:55.081135    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:36:55.083437 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:55.083416    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff2b27ad-c198-489c-9bb8-befeea447a64-metrics-certs\") pod \"network-metrics-daemon-fv6nk\" (UID: \"ff2b27ad-c198-489c-9bb8-befeea447a64\") " pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:36:55.150352 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:55.150321    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gv4zq\""
Sep 14 01:36:55.159157 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:55.159131    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fv6nk"
Sep 14 01:36:55.429341 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:55.429267    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fv6nk"]
Sep 14 01:36:55.432534 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:36:55.432507    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff2b27ad_c198_489c_9bb8_befeea447a64.slice/crio-58da65a56be485fd183f83cd8e6177a888510759e8a52b4a845b5409b6a7f022 WatchSource:0}: Error finding container 58da65a56be485fd183f83cd8e6177a888510759e8a52b4a845b5409b6a7f022: Status 404 returned error can't find the container with id 58da65a56be485fd183f83cd8e6177a888510759e8a52b4a845b5409b6a7f022
Sep 14 01:36:56.320960 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:56.320902    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fv6nk" event={"ID":"ff2b27ad-c198-489c-9bb8-befeea447a64","Type":"ContainerStarted","Data":"58da65a56be485fd183f83cd8e6177a888510759e8a52b4a845b5409b6a7f022"}
Sep 14 01:36:57.326615 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:57.326574    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fv6nk" event={"ID":"ff2b27ad-c198-489c-9bb8-befeea447a64","Type":"ContainerStarted","Data":"f3303a55c646a9d3f68eb2951135a1303a1e3835cff1d29e113cb50f1cf07152"}
Sep 14 01:36:57.327035 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:57.326628    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fv6nk" event={"ID":"ff2b27ad-c198-489c-9bb8-befeea447a64","Type":"ContainerStarted","Data":"2bc5142677ade0d829d41535f2137bd6cfae18f20b47cc8805991dc90509b712"}
Sep 14 01:36:57.341826 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:57.341770    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fv6nk" podStartSLOduration=375.327758432 podStartE2EDuration="6m16.341750126s" totalImagesPullingTime="1.013991694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:36:57.340841423 +0000 UTC m=+376.821292253" watchObservedRunningTime="2026-09-14 01:36:57.341750126 +0000 UTC m=+376.822200951"
Sep 14 01:36:58.362371 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.362335    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-658dd4fdff-4cx6n"]
Sep 14 01:36:58.362764 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.362682    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="202e1539-3daf-4627-b6c4-852744f3ddcb" containerName="console"
Sep 14 01:36:58.362764 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.362694    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="202e1539-3daf-4627-b6c4-852744f3ddcb" containerName="console"
Sep 14 01:36:58.362764 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.362708    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e585bb3-c6ca-49b9-a237-9a2971a73020" containerName="console"
Sep 14 01:36:58.362764 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.362714    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e585bb3-c6ca-49b9-a237-9a2971a73020" containerName="console"
Sep 14 01:36:58.362764 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.362724    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="526bc8e9-328b-4c29-912e-1e092a8b97fa" containerName="registry"
Sep 14 01:36:58.362764 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.362729    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="526bc8e9-328b-4c29-912e-1e092a8b97fa" containerName="registry"
Sep 14 01:36:58.363062 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.362775    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e585bb3-c6ca-49b9-a237-9a2971a73020" containerName="console"
Sep 14 01:36:58.363062 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.362784    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="202e1539-3daf-4627-b6c4-852744f3ddcb" containerName="console"
Sep 14 01:36:58.363062 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.362791    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="526bc8e9-328b-4c29-912e-1e092a8b97fa" containerName="registry"
Sep 14 01:36:58.365679 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.365664    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.376745 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.376713    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-658dd4fdff-4cx6n"]
Sep 14 01:36:58.403367 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.403337    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-config\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.403367 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.403368    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-trusted-ca-bundle\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.403599 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.403396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-oauth-serving-cert\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.403599 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.403469    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-serving-cert\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.403599 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.403505    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-service-ca\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.403599 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.403528    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42jmf\" (UniqueName: \"kubernetes.io/projected/16fcc77f-8774-43a4-bea9-e85ee7b46998-kube-api-access-42jmf\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.403599 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.403555    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-oauth-config\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.505110 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.505062    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-serving-cert\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.505306 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.505150    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-service-ca\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.505306 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.505186    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42jmf\" (UniqueName: \"kubernetes.io/projected/16fcc77f-8774-43a4-bea9-e85ee7b46998-kube-api-access-42jmf\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.505306 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.505222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-oauth-config\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.505450 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.505311    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-config\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.505450 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.505336    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-trusted-ca-bundle\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.505450 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.505381    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-oauth-serving-cert\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.506262 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.506233    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-oauth-serving-cert\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.509543 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.509355    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-serving-cert\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.510255 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.510227    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-service-ca\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.510373 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.510326    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-trusted-ca-bundle\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.511176 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.511150    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-config\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.511719 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.511694    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-oauth-config\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.516691 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.516669    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42jmf\" (UniqueName: \"kubernetes.io/projected/16fcc77f-8774-43a4-bea9-e85ee7b46998-kube-api-access-42jmf\") pod \"console-658dd4fdff-4cx6n\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") " pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.675425 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.675313    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:36:58.946706 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:58.946628    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-658dd4fdff-4cx6n"]
Sep 14 01:36:58.949642 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:36:58.949614    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16fcc77f_8774_43a4_bea9_e85ee7b46998.slice/crio-77f295d095b6cca05d8d97e3e963cffbaf07e516114e9827e6de7a2a80b100f7 WatchSource:0}: Error finding container 77f295d095b6cca05d8d97e3e963cffbaf07e516114e9827e6de7a2a80b100f7: Status 404 returned error can't find the container with id 77f295d095b6cca05d8d97e3e963cffbaf07e516114e9827e6de7a2a80b100f7
Sep 14 01:36:59.336357 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:59.336323    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658dd4fdff-4cx6n" event={"ID":"16fcc77f-8774-43a4-bea9-e85ee7b46998","Type":"ContainerStarted","Data":"af23404858d3f61a3197aa8e924e575bb4917e264d8d4ec77de57de9e0bed5a7"}
Sep 14 01:36:59.336540 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:59.336384    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658dd4fdff-4cx6n" event={"ID":"16fcc77f-8774-43a4-bea9-e85ee7b46998","Type":"ContainerStarted","Data":"77f295d095b6cca05d8d97e3e963cffbaf07e516114e9827e6de7a2a80b100f7"}
Sep 14 01:36:59.351690 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:36:59.351638    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-658dd4fdff-4cx6n" podStartSLOduration=1.351623574 podStartE2EDuration="1.351623574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:36:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:36:59.35049298 +0000 UTC m=+378.830943810" watchObservedRunningTime="2026-09-14 01:36:59.351623574 +0000 UTC m=+378.832074395"
Sep 14 01:37:08.675880 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:08.675838    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:37:08.676324 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:08.675951    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:37:08.680603 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:08.680577    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:37:09.377942 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:09.377898    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:37:09.423344 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:09.423312    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654bc7596-22qk9"]
Sep 14 01:37:34.442714 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.442674    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-654bc7596-22qk9" podUID="81b3b5c6-d14e-43cd-81cb-2af80cd60acf" containerName="console" containerID="cri-o://0f8f684387e011b7ed5a8a611b3a9572bad5d1d6d591806d03000a0e95b8b5c4" gracePeriod=15
Sep 14 01:37:34.689201 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.689174    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:37:34.814613 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.814571    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-oauth-config\") pod \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") "
Sep 14 01:37:34.814613 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.814619    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-service-ca\") pod \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") "
Sep 14 01:37:34.814871 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.814649    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-config\") pod \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") "
Sep 14 01:37:34.814871 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.814677    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-oauth-serving-cert\") pod \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") "
Sep 14 01:37:34.814871 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.814720    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-trusted-ca-bundle\") pod \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") "
Sep 14 01:37:34.814871 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.814753    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-serving-cert\") pod \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") "
Sep 14 01:37:34.814871 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.814838    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zf9s7\" (UniqueName: \"kubernetes.io/projected/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-kube-api-access-zf9s7\") pod \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\" (UID: \"81b3b5c6-d14e-43cd-81cb-2af80cd60acf\") "
Sep 14 01:37:34.815154 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.815108    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-service-ca" (OuterVolumeSpecName: "service-ca") pod "81b3b5c6-d14e-43cd-81cb-2af80cd60acf" (UID: "81b3b5c6-d14e-43cd-81cb-2af80cd60acf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:37:34.815208 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.815159    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "81b3b5c6-d14e-43cd-81cb-2af80cd60acf" (UID: "81b3b5c6-d14e-43cd-81cb-2af80cd60acf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:37:34.815208 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.815178    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-config" (OuterVolumeSpecName: "console-config") pod "81b3b5c6-d14e-43cd-81cb-2af80cd60acf" (UID: "81b3b5c6-d14e-43cd-81cb-2af80cd60acf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:37:34.815288 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.815209    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "81b3b5c6-d14e-43cd-81cb-2af80cd60acf" (UID: "81b3b5c6-d14e-43cd-81cb-2af80cd60acf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:37:34.816924 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.816893    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "81b3b5c6-d14e-43cd-81cb-2af80cd60acf" (UID: "81b3b5c6-d14e-43cd-81cb-2af80cd60acf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:37:34.817042 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.816974    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-kube-api-access-zf9s7" (OuterVolumeSpecName: "kube-api-access-zf9s7") pod "81b3b5c6-d14e-43cd-81cb-2af80cd60acf" (UID: "81b3b5c6-d14e-43cd-81cb-2af80cd60acf"). InnerVolumeSpecName "kube-api-access-zf9s7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:37:34.817090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.817064    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "81b3b5c6-d14e-43cd-81cb-2af80cd60acf" (UID: "81b3b5c6-d14e-43cd-81cb-2af80cd60acf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:37:34.915866 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.915821    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zf9s7\" (UniqueName: \"kubernetes.io/projected/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-kube-api-access-zf9s7\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:37:34.915866 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.915856    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-oauth-config\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:37:34.915866 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.915866    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:37:34.915866 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.915875    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-config\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:37:34.916156 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.915883    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-oauth-serving-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:37:34.916156 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.915892    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-trusted-ca-bundle\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:37:34.916156 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:34.915900    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81b3b5c6-d14e-43cd-81cb-2af80cd60acf-console-serving-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:37:35.483585 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:35.483549    2581 generic.go:356] "Generic (PLEG): container finished" podID="81b3b5c6-d14e-43cd-81cb-2af80cd60acf" containerID="0f8f684387e011b7ed5a8a611b3a9572bad5d1d6d591806d03000a0e95b8b5c4" exitCode=0
Sep 14 01:37:35.483585 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:35.483591    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654bc7596-22qk9" event={"ID":"81b3b5c6-d14e-43cd-81cb-2af80cd60acf","Type":"ContainerDied","Data":"0f8f684387e011b7ed5a8a611b3a9572bad5d1d6d591806d03000a0e95b8b5c4"}
Sep 14 01:37:35.484107 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:35.483623    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654bc7596-22qk9"
Sep 14 01:37:35.484107 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:35.483654    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654bc7596-22qk9" event={"ID":"81b3b5c6-d14e-43cd-81cb-2af80cd60acf","Type":"ContainerDied","Data":"ab2cd61959f59940ea6003b4758193e104b57b9b13ddd2f0a588fbff2669a9e2"}
Sep 14 01:37:35.484107 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:35.483673    2581 scope.go:117] "RemoveContainer" containerID="0f8f684387e011b7ed5a8a611b3a9572bad5d1d6d591806d03000a0e95b8b5c4"
Sep 14 01:37:35.492020 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:35.491968    2581 scope.go:117] "RemoveContainer" containerID="0f8f684387e011b7ed5a8a611b3a9572bad5d1d6d591806d03000a0e95b8b5c4"
Sep 14 01:37:35.492240 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:37:35.492223    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f8f684387e011b7ed5a8a611b3a9572bad5d1d6d591806d03000a0e95b8b5c4\": container with ID starting with 0f8f684387e011b7ed5a8a611b3a9572bad5d1d6d591806d03000a0e95b8b5c4 not found: ID does not exist" containerID="0f8f684387e011b7ed5a8a611b3a9572bad5d1d6d591806d03000a0e95b8b5c4"
Sep 14 01:37:35.492286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:35.492248    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f8f684387e011b7ed5a8a611b3a9572bad5d1d6d591806d03000a0e95b8b5c4"} err="failed to get container status \"0f8f684387e011b7ed5a8a611b3a9572bad5d1d6d591806d03000a0e95b8b5c4\": rpc error: code = NotFound desc = could not find container \"0f8f684387e011b7ed5a8a611b3a9572bad5d1d6d591806d03000a0e95b8b5c4\": container with ID starting with 0f8f684387e011b7ed5a8a611b3a9572bad5d1d6d591806d03000a0e95b8b5c4 not found: ID does not exist"
Sep 14 01:37:35.497843 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:35.497815    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654bc7596-22qk9"]
Sep 14 01:37:35.499368 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:35.499348    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-654bc7596-22qk9"]
Sep 14 01:37:37.052583 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:37.052538    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81b3b5c6-d14e-43cd-81cb-2af80cd60acf" path="/var/lib/kubelet/pods/81b3b5c6-d14e-43cd-81cb-2af80cd60acf/volumes"
Sep 14 01:37:52.396382 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.396338    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"]
Sep 14 01:37:52.396862 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.396833    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b3b5c6-d14e-43cd-81cb-2af80cd60acf" containerName="console"
Sep 14 01:37:52.396862 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.396850    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b3b5c6-d14e-43cd-81cb-2af80cd60acf" containerName="console"
Sep 14 01:37:52.396994 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.396958    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b3b5c6-d14e-43cd-81cb-2af80cd60acf" containerName="console"
Sep 14 01:37:52.401812 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.401789    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"
Sep 14 01:37:52.403919 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.403890    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 01:37:52.404052 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.404027    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tl294\""
Sep 14 01:37:52.404528 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.404513    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 01:37:52.408860 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.408830    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"]
Sep 14 01:37:52.459151 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.459113    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx\" (UID: \"ea6dfad7-3dbe-4f05-bb99-8855dc74b141\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"
Sep 14 01:37:52.459337 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.459158    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwbbh\" (UniqueName: \"kubernetes.io/projected/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-kube-api-access-qwbbh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx\" (UID: \"ea6dfad7-3dbe-4f05-bb99-8855dc74b141\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"
Sep 14 01:37:52.459337 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.459190    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx\" (UID: \"ea6dfad7-3dbe-4f05-bb99-8855dc74b141\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"
Sep 14 01:37:52.559882 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.559831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx\" (UID: \"ea6dfad7-3dbe-4f05-bb99-8855dc74b141\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"
Sep 14 01:37:52.559882 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.559891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwbbh\" (UniqueName: \"kubernetes.io/projected/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-kube-api-access-qwbbh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx\" (UID: \"ea6dfad7-3dbe-4f05-bb99-8855dc74b141\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"
Sep 14 01:37:52.560118 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.559944    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx\" (UID: \"ea6dfad7-3dbe-4f05-bb99-8855dc74b141\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"
Sep 14 01:37:52.560326 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.560306    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx\" (UID: \"ea6dfad7-3dbe-4f05-bb99-8855dc74b141\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"
Sep 14 01:37:52.560362 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.560312    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx\" (UID: \"ea6dfad7-3dbe-4f05-bb99-8855dc74b141\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"
Sep 14 01:37:52.567620 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.567586    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwbbh\" (UniqueName: \"kubernetes.io/projected/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-kube-api-access-qwbbh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx\" (UID: \"ea6dfad7-3dbe-4f05-bb99-8855dc74b141\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"
Sep 14 01:37:52.711723 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.711688    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"
Sep 14 01:37:52.984816 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:52.984756    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"]
Sep 14 01:37:52.987476 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:37:52.987439    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea6dfad7_3dbe_4f05_bb99_8855dc74b141.slice/crio-48c53cef98c227440ee59ab755bb3304fb43afd6c8ba0f554bfcf39cae96cafb WatchSource:0}: Error finding container 48c53cef98c227440ee59ab755bb3304fb43afd6c8ba0f554bfcf39cae96cafb: Status 404 returned error can't find the container with id 48c53cef98c227440ee59ab755bb3304fb43afd6c8ba0f554bfcf39cae96cafb
Sep 14 01:37:53.557862 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:53.557829    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx" event={"ID":"ea6dfad7-3dbe-4f05-bb99-8855dc74b141","Type":"ContainerStarted","Data":"48c53cef98c227440ee59ab755bb3304fb43afd6c8ba0f554bfcf39cae96cafb"}
Sep 14 01:37:59.585193 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:59.585101    2581 generic.go:356] "Generic (PLEG): container finished" podID="ea6dfad7-3dbe-4f05-bb99-8855dc74b141" containerID="38e4d0b1e1d0d4ec4da3cf41e8875ac4faf2b30a3f5d34f2b35979391405ecb1" exitCode=0
Sep 14 01:37:59.585193 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:37:59.585164    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx" event={"ID":"ea6dfad7-3dbe-4f05-bb99-8855dc74b141","Type":"ContainerDied","Data":"38e4d0b1e1d0d4ec4da3cf41e8875ac4faf2b30a3f5d34f2b35979391405ecb1"}
Sep 14 01:38:01.595540 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:01.595503    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx" event={"ID":"ea6dfad7-3dbe-4f05-bb99-8855dc74b141","Type":"ContainerStarted","Data":"534c2ea87a0200cb465e1e37613cc0fc9277ce4d54ab0d209d332f527c577b88"}
Sep 14 01:38:02.601270 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:02.601233    2581 generic.go:356] "Generic (PLEG): container finished" podID="ea6dfad7-3dbe-4f05-bb99-8855dc74b141" containerID="534c2ea87a0200cb465e1e37613cc0fc9277ce4d54ab0d209d332f527c577b88" exitCode=0
Sep 14 01:38:02.601653 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:02.601303    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx" event={"ID":"ea6dfad7-3dbe-4f05-bb99-8855dc74b141","Type":"ContainerDied","Data":"534c2ea87a0200cb465e1e37613cc0fc9277ce4d54ab0d209d332f527c577b88"}
Sep 14 01:38:09.634977 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:09.634934    2581 generic.go:356] "Generic (PLEG): container finished" podID="ea6dfad7-3dbe-4f05-bb99-8855dc74b141" containerID="ab24dec61bd7a6209760ba6a93066c26ea5e4204eafb6a62473eb6c0b68933c7" exitCode=0
Sep 14 01:38:09.634977 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:09.634947    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx" event={"ID":"ea6dfad7-3dbe-4f05-bb99-8855dc74b141","Type":"ContainerDied","Data":"ab24dec61bd7a6209760ba6a93066c26ea5e4204eafb6a62473eb6c0b68933c7"}
Sep 14 01:38:10.766833 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:10.766804    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"
Sep 14 01:38:10.833115 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:10.833077    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-util\") pod \"ea6dfad7-3dbe-4f05-bb99-8855dc74b141\" (UID: \"ea6dfad7-3dbe-4f05-bb99-8855dc74b141\") "
Sep 14 01:38:10.833299 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:10.833143    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-bundle\") pod \"ea6dfad7-3dbe-4f05-bb99-8855dc74b141\" (UID: \"ea6dfad7-3dbe-4f05-bb99-8855dc74b141\") "
Sep 14 01:38:10.833299 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:10.833212    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwbbh\" (UniqueName: \"kubernetes.io/projected/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-kube-api-access-qwbbh\") pod \"ea6dfad7-3dbe-4f05-bb99-8855dc74b141\" (UID: \"ea6dfad7-3dbe-4f05-bb99-8855dc74b141\") "
Sep 14 01:38:10.833839 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:10.833812    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-bundle" (OuterVolumeSpecName: "bundle") pod "ea6dfad7-3dbe-4f05-bb99-8855dc74b141" (UID: "ea6dfad7-3dbe-4f05-bb99-8855dc74b141"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:38:10.835466 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:10.835440    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-kube-api-access-qwbbh" (OuterVolumeSpecName: "kube-api-access-qwbbh") pod "ea6dfad7-3dbe-4f05-bb99-8855dc74b141" (UID: "ea6dfad7-3dbe-4f05-bb99-8855dc74b141"). InnerVolumeSpecName "kube-api-access-qwbbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:38:10.837265 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:10.837242    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-util" (OuterVolumeSpecName: "util") pod "ea6dfad7-3dbe-4f05-bb99-8855dc74b141" (UID: "ea6dfad7-3dbe-4f05-bb99-8855dc74b141"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:38:10.933884 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:10.933802    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwbbh\" (UniqueName: \"kubernetes.io/projected/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-kube-api-access-qwbbh\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:38:10.933884 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:10.933832    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-util\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:38:10.933884 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:10.933842    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea6dfad7-3dbe-4f05-bb99-8855dc74b141-bundle\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:38:11.645784 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:11.645748    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx" event={"ID":"ea6dfad7-3dbe-4f05-bb99-8855dc74b141","Type":"ContainerDied","Data":"48c53cef98c227440ee59ab755bb3304fb43afd6c8ba0f554bfcf39cae96cafb"}
Sep 14 01:38:11.645784 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:11.645780    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vzzrx"
Sep 14 01:38:11.646010 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:11.645823    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48c53cef98c227440ee59ab755bb3304fb43afd6c8ba0f554bfcf39cae96cafb"
Sep 14 01:38:15.347209 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.347175    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dxnxq"]
Sep 14 01:38:15.347582 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.347473    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea6dfad7-3dbe-4f05-bb99-8855dc74b141" containerName="extract"
Sep 14 01:38:15.347582 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.347483    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea6dfad7-3dbe-4f05-bb99-8855dc74b141" containerName="extract"
Sep 14 01:38:15.347582 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.347491    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea6dfad7-3dbe-4f05-bb99-8855dc74b141" containerName="pull"
Sep 14 01:38:15.347582 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.347496    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea6dfad7-3dbe-4f05-bb99-8855dc74b141" containerName="pull"
Sep 14 01:38:15.347582 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.347511    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea6dfad7-3dbe-4f05-bb99-8855dc74b141" containerName="util"
Sep 14 01:38:15.347582 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.347516    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea6dfad7-3dbe-4f05-bb99-8855dc74b141" containerName="util"
Sep 14 01:38:15.347582 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.347566    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea6dfad7-3dbe-4f05-bb99-8855dc74b141" containerName="extract"
Sep 14 01:38:15.350605 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.350584    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dxnxq"
Sep 14 01:38:15.352839 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.352812    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 14 01:38:15.352993 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.352872    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zxd8m\""
Sep 14 01:38:15.352993 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.352897    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 14 01:38:15.365059 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.365035    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dxnxq"]
Sep 14 01:38:15.468811 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.468770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v97kv\" (UniqueName: \"kubernetes.io/projected/3a12b97f-7ca0-44c7-ac8a-2af3dc64786d-kube-api-access-v97kv\") pod \"cert-manager-operator-controller-manager-55c444bf86-dxnxq\" (UID: \"3a12b97f-7ca0-44c7-ac8a-2af3dc64786d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dxnxq"
Sep 14 01:38:15.469018 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.468929    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a12b97f-7ca0-44c7-ac8a-2af3dc64786d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dxnxq\" (UID: \"3a12b97f-7ca0-44c7-ac8a-2af3dc64786d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dxnxq"
Sep 14 01:38:15.570382 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.570332    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a12b97f-7ca0-44c7-ac8a-2af3dc64786d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dxnxq\" (UID: \"3a12b97f-7ca0-44c7-ac8a-2af3dc64786d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dxnxq"
Sep 14 01:38:15.570382 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.570386    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v97kv\" (UniqueName: \"kubernetes.io/projected/3a12b97f-7ca0-44c7-ac8a-2af3dc64786d-kube-api-access-v97kv\") pod \"cert-manager-operator-controller-manager-55c444bf86-dxnxq\" (UID: \"3a12b97f-7ca0-44c7-ac8a-2af3dc64786d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dxnxq"
Sep 14 01:38:15.570746 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.570725    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a12b97f-7ca0-44c7-ac8a-2af3dc64786d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dxnxq\" (UID: \"3a12b97f-7ca0-44c7-ac8a-2af3dc64786d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dxnxq"
Sep 14 01:38:15.578235 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.578203    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v97kv\" (UniqueName: \"kubernetes.io/projected/3a12b97f-7ca0-44c7-ac8a-2af3dc64786d-kube-api-access-v97kv\") pod \"cert-manager-operator-controller-manager-55c444bf86-dxnxq\" (UID: \"3a12b97f-7ca0-44c7-ac8a-2af3dc64786d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dxnxq"
Sep 14 01:38:15.659313 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.659231    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dxnxq"
Sep 14 01:38:15.932304 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:15.932276    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dxnxq"]
Sep 14 01:38:15.935319 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:38:15.935282    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a12b97f_7ca0_44c7_ac8a_2af3dc64786d.slice/crio-5cb87aca6a495d61d959adc593a6c6b9ce5bdcfde5ae4a304d1c48980bdd5952 WatchSource:0}: Error finding container 5cb87aca6a495d61d959adc593a6c6b9ce5bdcfde5ae4a304d1c48980bdd5952: Status 404 returned error can't find the container with id 5cb87aca6a495d61d959adc593a6c6b9ce5bdcfde5ae4a304d1c48980bdd5952
Sep 14 01:38:16.669377 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:16.669333    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dxnxq" event={"ID":"3a12b97f-7ca0-44c7-ac8a-2af3dc64786d","Type":"ContainerStarted","Data":"5cb87aca6a495d61d959adc593a6c6b9ce5bdcfde5ae4a304d1c48980bdd5952"}
Sep 14 01:38:18.679430 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:18.679386    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dxnxq" event={"ID":"3a12b97f-7ca0-44c7-ac8a-2af3dc64786d","Type":"ContainerStarted","Data":"4d601a29cef0a4df84feca789ff2c4ecd028aaf06f693268ef9f921dd40c53bb"}
Sep 14 01:38:18.698626 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:18.698565    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dxnxq" podStartSLOduration=1.969819871 podStartE2EDuration="3.698547563s" totalImagesPullingTime="1.728727692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:38:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:38:18.696718 +0000 UTC m=+458.177168821" watchObservedRunningTime="2026-09-14 01:38:18.698547563 +0000 UTC m=+458.178998385"
Sep 14 01:38:19.643221 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.643186    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"]
Sep 14 01:38:19.645349 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.645335    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"
Sep 14 01:38:19.648447 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.648413    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 01:38:19.648541 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.648478    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 01:38:19.648605 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.648567    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tl294\""
Sep 14 01:38:19.658574 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.658549    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"]
Sep 14 01:38:19.702159 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.702104    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8a2c937-0986-4182-ac35-5c6be082b1a6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v\" (UID: \"c8a2c937-0986-4182-ac35-5c6be082b1a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"
Sep 14 01:38:19.702568 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.702187    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sslqj\" (UniqueName: \"kubernetes.io/projected/c8a2c937-0986-4182-ac35-5c6be082b1a6-kube-api-access-sslqj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v\" (UID: \"c8a2c937-0986-4182-ac35-5c6be082b1a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"
Sep 14 01:38:19.702568 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.702245    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8a2c937-0986-4182-ac35-5c6be082b1a6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v\" (UID: \"c8a2c937-0986-4182-ac35-5c6be082b1a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"
Sep 14 01:38:19.803579 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.803535    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8a2c937-0986-4182-ac35-5c6be082b1a6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v\" (UID: \"c8a2c937-0986-4182-ac35-5c6be082b1a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"
Sep 14 01:38:19.803786 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.803603    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sslqj\" (UniqueName: \"kubernetes.io/projected/c8a2c937-0986-4182-ac35-5c6be082b1a6-kube-api-access-sslqj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v\" (UID: \"c8a2c937-0986-4182-ac35-5c6be082b1a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"
Sep 14 01:38:19.803786 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.803659    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8a2c937-0986-4182-ac35-5c6be082b1a6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v\" (UID: \"c8a2c937-0986-4182-ac35-5c6be082b1a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"
Sep 14 01:38:19.804089 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.804066    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8a2c937-0986-4182-ac35-5c6be082b1a6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v\" (UID: \"c8a2c937-0986-4182-ac35-5c6be082b1a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"
Sep 14 01:38:19.804162 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.804100    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8a2c937-0986-4182-ac35-5c6be082b1a6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v\" (UID: \"c8a2c937-0986-4182-ac35-5c6be082b1a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"
Sep 14 01:38:19.810764 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.810740    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sslqj\" (UniqueName: \"kubernetes.io/projected/c8a2c937-0986-4182-ac35-5c6be082b1a6-kube-api-access-sslqj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v\" (UID: \"c8a2c937-0986-4182-ac35-5c6be082b1a6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"
Sep 14 01:38:19.954461 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:19.954430    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"
Sep 14 01:38:20.227858 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:20.227774    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"]
Sep 14 01:38:20.230046 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:38:20.230020    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8a2c937_0986_4182_ac35_5c6be082b1a6.slice/crio-095d327c49c1d4f92a979eacede82d6e4eefba8b85718ef6548ae033e3ecab10 WatchSource:0}: Error finding container 095d327c49c1d4f92a979eacede82d6e4eefba8b85718ef6548ae033e3ecab10: Status 404 returned error can't find the container with id 095d327c49c1d4f92a979eacede82d6e4eefba8b85718ef6548ae033e3ecab10
Sep 14 01:38:20.690479 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:20.690389    2581 generic.go:356] "Generic (PLEG): container finished" podID="c8a2c937-0986-4182-ac35-5c6be082b1a6" containerID="06dc9dfe9c4d0661da03f4bc336d282451539293e5a7168f7fdfc18ef8c3e68e" exitCode=0
Sep 14 01:38:20.690479 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:20.690466    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v" event={"ID":"c8a2c937-0986-4182-ac35-5c6be082b1a6","Type":"ContainerDied","Data":"06dc9dfe9c4d0661da03f4bc336d282451539293e5a7168f7fdfc18ef8c3e68e"}
Sep 14 01:38:20.690672 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:20.690513    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v" event={"ID":"c8a2c937-0986-4182-ac35-5c6be082b1a6","Type":"ContainerStarted","Data":"095d327c49c1d4f92a979eacede82d6e4eefba8b85718ef6548ae033e3ecab10"}
Sep 14 01:38:24.711226 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:24.711184    2581 generic.go:356] "Generic (PLEG): container finished" podID="c8a2c937-0986-4182-ac35-5c6be082b1a6" containerID="ca5eae462d1f91231c5be11c9f6a18671c4f703ca26117ccc3fd7597d65c4b6b" exitCode=0
Sep 14 01:38:24.711583 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:24.711261    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v" event={"ID":"c8a2c937-0986-4182-ac35-5c6be082b1a6","Type":"ContainerDied","Data":"ca5eae462d1f91231c5be11c9f6a18671c4f703ca26117ccc3fd7597d65c4b6b"}
Sep 14 01:38:25.399821 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.399788    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9k2cc"]
Sep 14 01:38:25.401811 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.401793    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9k2cc"
Sep 14 01:38:25.403992 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.403963    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 14 01:38:25.404695 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.404678    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 14 01:38:25.404794 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.404705    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-4pvhn\""
Sep 14 01:38:25.410896 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.410872    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9k2cc"]
Sep 14 01:38:25.554719 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.554674    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjhd5\" (UniqueName: \"kubernetes.io/projected/4cf6c198-7988-4b90-91d3-9d827b7fd8e9-kube-api-access-mjhd5\") pod \"cert-manager-webhook-6486ffd997-9k2cc\" (UID: \"4cf6c198-7988-4b90-91d3-9d827b7fd8e9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9k2cc"
Sep 14 01:38:25.554896 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.554788    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cf6c198-7988-4b90-91d3-9d827b7fd8e9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9k2cc\" (UID: \"4cf6c198-7988-4b90-91d3-9d827b7fd8e9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9k2cc"
Sep 14 01:38:25.655783 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.655700    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cf6c198-7988-4b90-91d3-9d827b7fd8e9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9k2cc\" (UID: \"4cf6c198-7988-4b90-91d3-9d827b7fd8e9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9k2cc"
Sep 14 01:38:25.655783 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.655748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjhd5\" (UniqueName: \"kubernetes.io/projected/4cf6c198-7988-4b90-91d3-9d827b7fd8e9-kube-api-access-mjhd5\") pod \"cert-manager-webhook-6486ffd997-9k2cc\" (UID: \"4cf6c198-7988-4b90-91d3-9d827b7fd8e9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9k2cc"
Sep 14 01:38:25.663326 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.663295    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjhd5\" (UniqueName: \"kubernetes.io/projected/4cf6c198-7988-4b90-91d3-9d827b7fd8e9-kube-api-access-mjhd5\") pod \"cert-manager-webhook-6486ffd997-9k2cc\" (UID: \"4cf6c198-7988-4b90-91d3-9d827b7fd8e9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9k2cc"
Sep 14 01:38:25.663456 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.663330    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cf6c198-7988-4b90-91d3-9d827b7fd8e9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9k2cc\" (UID: \"4cf6c198-7988-4b90-91d3-9d827b7fd8e9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9k2cc"
Sep 14 01:38:25.717679 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.717645    2581 generic.go:356] "Generic (PLEG): container finished" podID="c8a2c937-0986-4182-ac35-5c6be082b1a6" containerID="079b11c9d5cbf8be65c93d3983b0227afd435d03ebc766649a85da37a5c0f759" exitCode=0
Sep 14 01:38:25.718036 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.717725    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v" event={"ID":"c8a2c937-0986-4182-ac35-5c6be082b1a6","Type":"ContainerDied","Data":"079b11c9d5cbf8be65c93d3983b0227afd435d03ebc766649a85da37a5c0f759"}
Sep 14 01:38:25.721873 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.721858    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9k2cc"
Sep 14 01:38:25.992755 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:25.992721    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9k2cc"]
Sep 14 01:38:25.995665 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:38:25.995636    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cf6c198_7988_4b90_91d3_9d827b7fd8e9.slice/crio-c63695902a4e21414c6a9bd75f965d57971d436ce192a81b643da31a2a5060fc WatchSource:0}: Error finding container c63695902a4e21414c6a9bd75f965d57971d436ce192a81b643da31a2a5060fc: Status 404 returned error can't find the container with id c63695902a4e21414c6a9bd75f965d57971d436ce192a81b643da31a2a5060fc
Sep 14 01:38:26.723446 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:26.723410    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9k2cc" event={"ID":"4cf6c198-7988-4b90-91d3-9d827b7fd8e9","Type":"ContainerStarted","Data":"c63695902a4e21414c6a9bd75f965d57971d436ce192a81b643da31a2a5060fc"}
Sep 14 01:38:26.847637 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:26.847611    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"
Sep 14 01:38:26.966378 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:26.966337    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8a2c937-0986-4182-ac35-5c6be082b1a6-util\") pod \"c8a2c937-0986-4182-ac35-5c6be082b1a6\" (UID: \"c8a2c937-0986-4182-ac35-5c6be082b1a6\") "
Sep 14 01:38:26.966573 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:26.966449    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8a2c937-0986-4182-ac35-5c6be082b1a6-bundle\") pod \"c8a2c937-0986-4182-ac35-5c6be082b1a6\" (UID: \"c8a2c937-0986-4182-ac35-5c6be082b1a6\") "
Sep 14 01:38:26.966573 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:26.966483    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sslqj\" (UniqueName: \"kubernetes.io/projected/c8a2c937-0986-4182-ac35-5c6be082b1a6-kube-api-access-sslqj\") pod \"c8a2c937-0986-4182-ac35-5c6be082b1a6\" (UID: \"c8a2c937-0986-4182-ac35-5c6be082b1a6\") "
Sep 14 01:38:26.966826 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:26.966795    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8a2c937-0986-4182-ac35-5c6be082b1a6-bundle" (OuterVolumeSpecName: "bundle") pod "c8a2c937-0986-4182-ac35-5c6be082b1a6" (UID: "c8a2c937-0986-4182-ac35-5c6be082b1a6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:38:26.968680 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:26.968656    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8a2c937-0986-4182-ac35-5c6be082b1a6-kube-api-access-sslqj" (OuterVolumeSpecName: "kube-api-access-sslqj") pod "c8a2c937-0986-4182-ac35-5c6be082b1a6" (UID: "c8a2c937-0986-4182-ac35-5c6be082b1a6"). InnerVolumeSpecName "kube-api-access-sslqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:38:26.971321 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:26.971284    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8a2c937-0986-4182-ac35-5c6be082b1a6-util" (OuterVolumeSpecName: "util") pod "c8a2c937-0986-4182-ac35-5c6be082b1a6" (UID: "c8a2c937-0986-4182-ac35-5c6be082b1a6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:38:27.067483 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:27.067456    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8a2c937-0986-4182-ac35-5c6be082b1a6-bundle\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:38:27.067483 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:27.067482    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sslqj\" (UniqueName: \"kubernetes.io/projected/c8a2c937-0986-4182-ac35-5c6be082b1a6-kube-api-access-sslqj\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:38:27.067662 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:27.067495    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8a2c937-0986-4182-ac35-5c6be082b1a6-util\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:38:27.729665 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:27.729635    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v"
Sep 14 01:38:27.730106 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:27.729635    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rj2v" event={"ID":"c8a2c937-0986-4182-ac35-5c6be082b1a6","Type":"ContainerDied","Data":"095d327c49c1d4f92a979eacede82d6e4eefba8b85718ef6548ae033e3ecab10"}
Sep 14 01:38:27.730106 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:27.729781    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="095d327c49c1d4f92a979eacede82d6e4eefba8b85718ef6548ae033e3ecab10"
Sep 14 01:38:31.258343 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.258309    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xrjr2"]
Sep 14 01:38:31.258840 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.258752    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8a2c937-0986-4182-ac35-5c6be082b1a6" containerName="extract"
Sep 14 01:38:31.258840 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.258772    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8a2c937-0986-4182-ac35-5c6be082b1a6" containerName="extract"
Sep 14 01:38:31.258840 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.258784    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8a2c937-0986-4182-ac35-5c6be082b1a6" containerName="util"
Sep 14 01:38:31.258840 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.258793    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8a2c937-0986-4182-ac35-5c6be082b1a6" containerName="util"
Sep 14 01:38:31.258840 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.258820    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8a2c937-0986-4182-ac35-5c6be082b1a6" containerName="pull"
Sep 14 01:38:31.258840 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.258829    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8a2c937-0986-4182-ac35-5c6be082b1a6" containerName="pull"
Sep 14 01:38:31.259170 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.258935    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8a2c937-0986-4182-ac35-5c6be082b1a6" containerName="extract"
Sep 14 01:38:31.262627 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.262606    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xrjr2"
Sep 14 01:38:31.264461 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.264440    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-xcvt9\""
Sep 14 01:38:31.270537 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.270514    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xrjr2"]
Sep 14 01:38:31.412090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.412053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xh8m\" (UniqueName: \"kubernetes.io/projected/5eb590ba-0e62-48be-a5a7-0dbf479ca8a7-kube-api-access-5xh8m\") pod \"cert-manager-55b4d954d5-xrjr2\" (UID: \"5eb590ba-0e62-48be-a5a7-0dbf479ca8a7\") " pod="cert-manager/cert-manager-55b4d954d5-xrjr2"
Sep 14 01:38:31.412265 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.412103    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5eb590ba-0e62-48be-a5a7-0dbf479ca8a7-bound-sa-token\") pod \"cert-manager-55b4d954d5-xrjr2\" (UID: \"5eb590ba-0e62-48be-a5a7-0dbf479ca8a7\") " pod="cert-manager/cert-manager-55b4d954d5-xrjr2"
Sep 14 01:38:31.512634 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.512541    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xh8m\" (UniqueName: \"kubernetes.io/projected/5eb590ba-0e62-48be-a5a7-0dbf479ca8a7-kube-api-access-5xh8m\") pod \"cert-manager-55b4d954d5-xrjr2\" (UID: \"5eb590ba-0e62-48be-a5a7-0dbf479ca8a7\") " pod="cert-manager/cert-manager-55b4d954d5-xrjr2"
Sep 14 01:38:31.512634 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.512586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5eb590ba-0e62-48be-a5a7-0dbf479ca8a7-bound-sa-token\") pod \"cert-manager-55b4d954d5-xrjr2\" (UID: \"5eb590ba-0e62-48be-a5a7-0dbf479ca8a7\") " pod="cert-manager/cert-manager-55b4d954d5-xrjr2"
Sep 14 01:38:31.519470 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.519436    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5eb590ba-0e62-48be-a5a7-0dbf479ca8a7-bound-sa-token\") pod \"cert-manager-55b4d954d5-xrjr2\" (UID: \"5eb590ba-0e62-48be-a5a7-0dbf479ca8a7\") " pod="cert-manager/cert-manager-55b4d954d5-xrjr2"
Sep 14 01:38:31.519641 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.519620    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xh8m\" (UniqueName: \"kubernetes.io/projected/5eb590ba-0e62-48be-a5a7-0dbf479ca8a7-kube-api-access-5xh8m\") pod \"cert-manager-55b4d954d5-xrjr2\" (UID: \"5eb590ba-0e62-48be-a5a7-0dbf479ca8a7\") " pod="cert-manager/cert-manager-55b4d954d5-xrjr2"
Sep 14 01:38:31.572174 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.572123    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xrjr2"
Sep 14 01:38:31.847173 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:31.847120    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xrjr2"]
Sep 14 01:38:31.849641 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:38:31.849612    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5eb590ba_0e62_48be_a5a7_0dbf479ca8a7.slice/crio-9ac730a105a0b6ec0e433363d84de8ad7ca48d0d3e32ef649cb68204b48e377d WatchSource:0}: Error finding container 9ac730a105a0b6ec0e433363d84de8ad7ca48d0d3e32ef649cb68204b48e377d: Status 404 returned error can't find the container with id 9ac730a105a0b6ec0e433363d84de8ad7ca48d0d3e32ef649cb68204b48e377d
Sep 14 01:38:32.753950 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:32.753889    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xrjr2" event={"ID":"5eb590ba-0e62-48be-a5a7-0dbf479ca8a7","Type":"ContainerStarted","Data":"9ac730a105a0b6ec0e433363d84de8ad7ca48d0d3e32ef649cb68204b48e377d"}
Sep 14 01:38:35.769615 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:35.769572    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9k2cc" event={"ID":"4cf6c198-7988-4b90-91d3-9d827b7fd8e9","Type":"ContainerStarted","Data":"19b641f3c432df5f072911ca7842ebf4006f5fa3afa8d904ea1f73e72c33475a"}
Sep 14 01:38:35.770017 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:35.769732    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9k2cc"
Sep 14 01:38:35.771044 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:35.771021    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xrjr2" event={"ID":"5eb590ba-0e62-48be-a5a7-0dbf479ca8a7","Type":"ContainerStarted","Data":"b2bd5485a87e4f2eb1f49923ff74e686aea969e6b92500eb61152d2bdf5e8a2b"}
Sep 14 01:38:35.785563 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:35.785512    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-9k2cc" podStartSLOduration=2.0421612160000002 podStartE2EDuration="10.785496909s" totalImagesPullingTime="8.743335693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:38:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:38:35.783449669 +0000 UTC m=+475.263900489" watchObservedRunningTime="2026-09-14 01:38:35.785496909 +0000 UTC m=+475.265947731"
Sep 14 01:38:35.797239 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:35.797182    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-xrjr2" podStartSLOduration=1.9010089300000002 podStartE2EDuration="4.797161717s" totalImagesPullingTime="2.896152787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:38:35.797002335 +0000 UTC m=+475.277453157" watchObservedRunningTime="2026-09-14 01:38:35.797161717 +0000 UTC m=+475.277612539"
Sep 14 01:38:40.395583 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.395545    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"]
Sep 14 01:38:40.398780 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.398761    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"
Sep 14 01:38:40.400972 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.400946    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 01:38:40.401101 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.400958    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 01:38:40.401904 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.401885    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tl294\""
Sep 14 01:38:40.406964 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.406940    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"]
Sep 14 01:38:40.490460 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.490423    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fqnp\" (UniqueName: \"kubernetes.io/projected/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-kube-api-access-7fqnp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p\" (UID: \"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"
Sep 14 01:38:40.490661 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.490502    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p\" (UID: \"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"
Sep 14 01:38:40.490661 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.490541    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p\" (UID: \"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"
Sep 14 01:38:40.591363 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.591320    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p\" (UID: \"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"
Sep 14 01:38:40.591363 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.591369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p\" (UID: \"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"
Sep 14 01:38:40.591627 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.591407    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fqnp\" (UniqueName: \"kubernetes.io/projected/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-kube-api-access-7fqnp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p\" (UID: \"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"
Sep 14 01:38:40.591744 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.591722    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p\" (UID: \"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"
Sep 14 01:38:40.591808 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.591756    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p\" (UID: \"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"
Sep 14 01:38:40.598747 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.598714    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fqnp\" (UniqueName: \"kubernetes.io/projected/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-kube-api-access-7fqnp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p\" (UID: \"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"
Sep 14 01:38:40.708531 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:40.708503    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"
Sep 14 01:38:41.021066 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:41.020997    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"]
Sep 14 01:38:41.023207 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:38:41.023178    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b0c4ff_0bf7_4591_92b8_5ee5ca5f9656.slice/crio-ee77385e905ddc6b3493bc5f73f5f3df093e7794c01f0e97faa105269a745e48 WatchSource:0}: Error finding container ee77385e905ddc6b3493bc5f73f5f3df093e7794c01f0e97faa105269a745e48: Status 404 returned error can't find the container with id ee77385e905ddc6b3493bc5f73f5f3df093e7794c01f0e97faa105269a745e48
Sep 14 01:38:41.777643 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:41.777606    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9k2cc"
Sep 14 01:38:41.800274 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:41.800238    2581 generic.go:356] "Generic (PLEG): container finished" podID="40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656" containerID="e8de1616db73841fee19df16e18d3cd9b914bd8e71f33fa5a34314d0f5d79eca" exitCode=0
Sep 14 01:38:41.800452 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:41.800283    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p" event={"ID":"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656","Type":"ContainerDied","Data":"e8de1616db73841fee19df16e18d3cd9b914bd8e71f33fa5a34314d0f5d79eca"}
Sep 14 01:38:41.800452 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:41.800328    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p" event={"ID":"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656","Type":"ContainerStarted","Data":"ee77385e905ddc6b3493bc5f73f5f3df093e7794c01f0e97faa105269a745e48"}
Sep 14 01:38:43.811146 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:43.811106    2581 generic.go:356] "Generic (PLEG): container finished" podID="40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656" containerID="9c094fb92f9036abc42f4d219cd9612639c6f44dfbff74d5d519efb3bbdfadcf" exitCode=0
Sep 14 01:38:43.811621 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:43.811195    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p" event={"ID":"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656","Type":"ContainerDied","Data":"9c094fb92f9036abc42f4d219cd9612639c6f44dfbff74d5d519efb3bbdfadcf"}
Sep 14 01:38:44.818279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:44.818237    2581 generic.go:356] "Generic (PLEG): container finished" podID="40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656" containerID="60f9906b3d0559af2083a45d2296b67f4a1103f3504de2caf47615a841304ed8" exitCode=0
Sep 14 01:38:44.818658 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:44.818316    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p" event={"ID":"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656","Type":"ContainerDied","Data":"60f9906b3d0559af2083a45d2296b67f4a1103f3504de2caf47615a841304ed8"}
Sep 14 01:38:45.967448 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:45.967417    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"
Sep 14 01:38:46.140253 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:46.140174    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fqnp\" (UniqueName: \"kubernetes.io/projected/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-kube-api-access-7fqnp\") pod \"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656\" (UID: \"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656\") "
Sep 14 01:38:46.140253 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:46.140214    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-bundle\") pod \"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656\" (UID: \"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656\") "
Sep 14 01:38:46.140253 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:46.140239    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-util\") pod \"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656\" (UID: \"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656\") "
Sep 14 01:38:46.140964 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:46.140927    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-bundle" (OuterVolumeSpecName: "bundle") pod "40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656" (UID: "40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:38:46.142350 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:46.142319    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-kube-api-access-7fqnp" (OuterVolumeSpecName: "kube-api-access-7fqnp") pod "40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656" (UID: "40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656"). InnerVolumeSpecName "kube-api-access-7fqnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:38:46.146479 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:46.146454    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-util" (OuterVolumeSpecName: "util") pod "40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656" (UID: "40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:38:46.241521 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:46.241482    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fqnp\" (UniqueName: \"kubernetes.io/projected/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-kube-api-access-7fqnp\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:38:46.241521 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:46.241526    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-bundle\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:38:46.241521 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:46.241534    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656-util\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:38:46.829453 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:46.829417    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p" event={"ID":"40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656","Type":"ContainerDied","Data":"ee77385e905ddc6b3493bc5f73f5f3df093e7794c01f0e97faa105269a745e48"}
Sep 14 01:38:46.829453 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:46.829444    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpr59p"
Sep 14 01:38:46.829665 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:46.829472    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee77385e905ddc6b3493bc5f73f5f3df093e7794c01f0e97faa105269a745e48"
Sep 14 01:38:57.416659 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.416620    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"]
Sep 14 01:38:57.417437 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.416966    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656" containerName="extract"
Sep 14 01:38:57.417437 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.416981    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656" containerName="extract"
Sep 14 01:38:57.417437 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.416993    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656" containerName="util"
Sep 14 01:38:57.417437 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.416999    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656" containerName="util"
Sep 14 01:38:57.417437 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.417005    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656" containerName="pull"
Sep 14 01:38:57.417437 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.417011    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656" containerName="pull"
Sep 14 01:38:57.417437 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.417083    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="40b0c4ff-0bf7-4591-92b8-5ee5ca5f9656" containerName="extract"
Sep 14 01:38:57.419028 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.419007    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"
Sep 14 01:38:57.421153 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.421129    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 01:38:57.421287 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.421224    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 01:38:57.421287 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.421222    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tl294\""
Sep 14 01:38:57.428221 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.428199    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54d3c703-27b7-486a-b619-001ee40d498f-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf\" (UID: \"54d3c703-27b7-486a-b619-001ee40d498f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"
Sep 14 01:38:57.428318 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.428260    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54d3c703-27b7-486a-b619-001ee40d498f-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf\" (UID: \"54d3c703-27b7-486a-b619-001ee40d498f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"
Sep 14 01:38:57.428318 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.428300    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-426kk\" (UniqueName: \"kubernetes.io/projected/54d3c703-27b7-486a-b619-001ee40d498f-kube-api-access-426kk\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf\" (UID: \"54d3c703-27b7-486a-b619-001ee40d498f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"
Sep 14 01:38:57.431303 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.431282    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"]
Sep 14 01:38:57.522510 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.522474    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"]
Sep 14 01:38:57.524716 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.524698    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.527511 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.527489    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 14 01:38:57.527511 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.527503    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kpfz9\""
Sep 14 01:38:57.527883 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.527869    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 14 01:38:57.528319 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.528306    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 14 01:38:57.528455 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.528441    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 14 01:38:57.528702 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.528688    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-426kk\" (UniqueName: \"kubernetes.io/projected/54d3c703-27b7-486a-b619-001ee40d498f-kube-api-access-426kk\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf\" (UID: \"54d3c703-27b7-486a-b619-001ee40d498f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"
Sep 14 01:38:57.528754 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.528731    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2674dd1c-d5b3-427c-9386-bc375c7268e7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c5d46b944-lbcsc\" (UID: \"2674dd1c-d5b3-427c-9386-bc375c7268e7\") " pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.528754 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.528749    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8rdc\" (UniqueName: \"kubernetes.io/projected/2674dd1c-d5b3-427c-9386-bc375c7268e7-kube-api-access-m8rdc\") pod \"opendatahub-operator-controller-manager-c5d46b944-lbcsc\" (UID: \"2674dd1c-d5b3-427c-9386-bc375c7268e7\") " pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.528898 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.528880    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54d3c703-27b7-486a-b619-001ee40d498f-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf\" (UID: \"54d3c703-27b7-486a-b619-001ee40d498f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"
Sep 14 01:38:57.528986 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.528928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2674dd1c-d5b3-427c-9386-bc375c7268e7-tmp\") pod \"opendatahub-operator-controller-manager-c5d46b944-lbcsc\" (UID: \"2674dd1c-d5b3-427c-9386-bc375c7268e7\") " pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.528986 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.528967    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2674dd1c-d5b3-427c-9386-bc375c7268e7-webhook-cert\") pod \"opendatahub-operator-controller-manager-c5d46b944-lbcsc\" (UID: \"2674dd1c-d5b3-427c-9386-bc375c7268e7\") " pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.529079 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.528997    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2674dd1c-d5b3-427c-9386-bc375c7268e7-manifests\") pod \"opendatahub-operator-controller-manager-c5d46b944-lbcsc\" (UID: \"2674dd1c-d5b3-427c-9386-bc375c7268e7\") " pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.529079 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.529057    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54d3c703-27b7-486a-b619-001ee40d498f-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf\" (UID: \"54d3c703-27b7-486a-b619-001ee40d498f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"
Sep 14 01:38:57.529288 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.529267    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54d3c703-27b7-486a-b619-001ee40d498f-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf\" (UID: \"54d3c703-27b7-486a-b619-001ee40d498f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"
Sep 14 01:38:57.529346 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.529331    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54d3c703-27b7-486a-b619-001ee40d498f-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf\" (UID: \"54d3c703-27b7-486a-b619-001ee40d498f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"
Sep 14 01:38:57.539829 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.539801    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-426kk\" (UniqueName: \"kubernetes.io/projected/54d3c703-27b7-486a-b619-001ee40d498f-kube-api-access-426kk\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf\" (UID: \"54d3c703-27b7-486a-b619-001ee40d498f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"
Sep 14 01:38:57.553555 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.553523    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"]
Sep 14 01:38:57.630051 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.630017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2674dd1c-d5b3-427c-9386-bc375c7268e7-tmp\") pod \"opendatahub-operator-controller-manager-c5d46b944-lbcsc\" (UID: \"2674dd1c-d5b3-427c-9386-bc375c7268e7\") " pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.630051 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.630056    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2674dd1c-d5b3-427c-9386-bc375c7268e7-webhook-cert\") pod \"opendatahub-operator-controller-manager-c5d46b944-lbcsc\" (UID: \"2674dd1c-d5b3-427c-9386-bc375c7268e7\") " pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.630297 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.630078    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2674dd1c-d5b3-427c-9386-bc375c7268e7-manifests\") pod \"opendatahub-operator-controller-manager-c5d46b944-lbcsc\" (UID: \"2674dd1c-d5b3-427c-9386-bc375c7268e7\") " pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.630297 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.630129    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2674dd1c-d5b3-427c-9386-bc375c7268e7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c5d46b944-lbcsc\" (UID: \"2674dd1c-d5b3-427c-9386-bc375c7268e7\") " pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.630297 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.630145    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8rdc\" (UniqueName: \"kubernetes.io/projected/2674dd1c-d5b3-427c-9386-bc375c7268e7-kube-api-access-m8rdc\") pod \"opendatahub-operator-controller-manager-c5d46b944-lbcsc\" (UID: \"2674dd1c-d5b3-427c-9386-bc375c7268e7\") " pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.630564 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.630542    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2674dd1c-d5b3-427c-9386-bc375c7268e7-manifests\") pod \"opendatahub-operator-controller-manager-c5d46b944-lbcsc\" (UID: \"2674dd1c-d5b3-427c-9386-bc375c7268e7\") " pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.632510 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.632484    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2674dd1c-d5b3-427c-9386-bc375c7268e7-tmp\") pod \"opendatahub-operator-controller-manager-c5d46b944-lbcsc\" (UID: \"2674dd1c-d5b3-427c-9386-bc375c7268e7\") " pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.632749 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.632727    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2674dd1c-d5b3-427c-9386-bc375c7268e7-webhook-cert\") pod \"opendatahub-operator-controller-manager-c5d46b944-lbcsc\" (UID: \"2674dd1c-d5b3-427c-9386-bc375c7268e7\") " pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.632801 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.632787    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2674dd1c-d5b3-427c-9386-bc375c7268e7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c5d46b944-lbcsc\" (UID: \"2674dd1c-d5b3-427c-9386-bc375c7268e7\") " pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.644036 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.644008    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8rdc\" (UniqueName: \"kubernetes.io/projected/2674dd1c-d5b3-427c-9386-bc375c7268e7-kube-api-access-m8rdc\") pod \"opendatahub-operator-controller-manager-c5d46b944-lbcsc\" (UID: \"2674dd1c-d5b3-427c-9386-bc375c7268e7\") " pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.728375 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.728334    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"
Sep 14 01:38:57.835182 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.835140    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:38:57.999395 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:57.999363    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"]
Sep 14 01:38:58.001718 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:38:58.001688    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d3c703_27b7_486a_b619_001ee40d498f.slice/crio-84d8a8d74560e6f6526cb03ba7d8a9b70ad9b59b3bd6ecb5f62e83437e3f035b WatchSource:0}: Error finding container 84d8a8d74560e6f6526cb03ba7d8a9b70ad9b59b3bd6ecb5f62e83437e3f035b: Status 404 returned error can't find the container with id 84d8a8d74560e6f6526cb03ba7d8a9b70ad9b59b3bd6ecb5f62e83437e3f035b
Sep 14 01:38:58.109745 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:58.109716    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"]
Sep 14 01:38:58.162834 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:38:58.162805    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2674dd1c_d5b3_427c_9386_bc375c7268e7.slice/crio-421efc9f8d1c6b86fb0a31962bdcd300b9fa6c6c99fd5ab5cccb628f1d72e5b5 WatchSource:0}: Error finding container 421efc9f8d1c6b86fb0a31962bdcd300b9fa6c6c99fd5ab5cccb628f1d72e5b5: Status 404 returned error can't find the container with id 421efc9f8d1c6b86fb0a31962bdcd300b9fa6c6c99fd5ab5cccb628f1d72e5b5
Sep 14 01:38:58.891852 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:58.891810    2581 generic.go:356] "Generic (PLEG): container finished" podID="54d3c703-27b7-486a-b619-001ee40d498f" containerID="c149797e2452f43ed961c84f9c383607776d9934d0f52541dcd989ad7464d1ef" exitCode=0
Sep 14 01:38:58.892339 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:58.891905    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf" event={"ID":"54d3c703-27b7-486a-b619-001ee40d498f","Type":"ContainerDied","Data":"c149797e2452f43ed961c84f9c383607776d9934d0f52541dcd989ad7464d1ef"}
Sep 14 01:38:58.892339 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:58.892021    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf" event={"ID":"54d3c703-27b7-486a-b619-001ee40d498f","Type":"ContainerStarted","Data":"84d8a8d74560e6f6526cb03ba7d8a9b70ad9b59b3bd6ecb5f62e83437e3f035b"}
Sep 14 01:38:58.893834 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:58.893804    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc" event={"ID":"2674dd1c-d5b3-427c-9386-bc375c7268e7","Type":"ContainerStarted","Data":"421efc9f8d1c6b86fb0a31962bdcd300b9fa6c6c99fd5ab5cccb628f1d72e5b5"}
Sep 14 01:38:59.902875 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:38:59.902832    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf" event={"ID":"54d3c703-27b7-486a-b619-001ee40d498f","Type":"ContainerStarted","Data":"d77bd20f0a69ff52e1351db4439089a71a30af7d7075b6492526a54470b46489"}
Sep 14 01:39:00.911173 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:00.911134    2581 generic.go:356] "Generic (PLEG): container finished" podID="54d3c703-27b7-486a-b619-001ee40d498f" containerID="d77bd20f0a69ff52e1351db4439089a71a30af7d7075b6492526a54470b46489" exitCode=0
Sep 14 01:39:00.911578 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:00.911233    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf" event={"ID":"54d3c703-27b7-486a-b619-001ee40d498f","Type":"ContainerDied","Data":"d77bd20f0a69ff52e1351db4439089a71a30af7d7075b6492526a54470b46489"}
Sep 14 01:39:01.917837 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:01.917799    2581 generic.go:356] "Generic (PLEG): container finished" podID="54d3c703-27b7-486a-b619-001ee40d498f" containerID="b536416992050c3f1f43498ca0de559b5470f24bc4660a60f8925cf4588a0119" exitCode=0
Sep 14 01:39:01.918313 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:01.917877    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf" event={"ID":"54d3c703-27b7-486a-b619-001ee40d498f","Type":"ContainerDied","Data":"b536416992050c3f1f43498ca0de559b5470f24bc4660a60f8925cf4588a0119"}
Sep 14 01:39:01.919619 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:01.919598    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc" event={"ID":"2674dd1c-d5b3-427c-9386-bc375c7268e7","Type":"ContainerStarted","Data":"3f83304dde62429b2de145ecdc796c726facb5a0933659cf52b8d9190314cdce"}
Sep 14 01:39:02.927320 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:02.927284    2581 generic.go:356] "Generic (PLEG): container finished" podID="2674dd1c-d5b3-427c-9386-bc375c7268e7" containerID="3f83304dde62429b2de145ecdc796c726facb5a0933659cf52b8d9190314cdce" exitCode=0
Sep 14 01:39:02.927786 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:02.927379    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc" event={"ID":"2674dd1c-d5b3-427c-9386-bc375c7268e7","Type":"ContainerDied","Data":"3f83304dde62429b2de145ecdc796c726facb5a0933659cf52b8d9190314cdce"}
Sep 14 01:39:03.059575 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.059551    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"
Sep 14 01:39:03.077581 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.077551    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54d3c703-27b7-486a-b619-001ee40d498f-bundle\") pod \"54d3c703-27b7-486a-b619-001ee40d498f\" (UID: \"54d3c703-27b7-486a-b619-001ee40d498f\") "
Sep 14 01:39:03.077740 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.077606    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54d3c703-27b7-486a-b619-001ee40d498f-util\") pod \"54d3c703-27b7-486a-b619-001ee40d498f\" (UID: \"54d3c703-27b7-486a-b619-001ee40d498f\") "
Sep 14 01:39:03.077740 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.077671    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-426kk\" (UniqueName: \"kubernetes.io/projected/54d3c703-27b7-486a-b619-001ee40d498f-kube-api-access-426kk\") pod \"54d3c703-27b7-486a-b619-001ee40d498f\" (UID: \"54d3c703-27b7-486a-b619-001ee40d498f\") "
Sep 14 01:39:03.078431 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.078405    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d3c703-27b7-486a-b619-001ee40d498f-bundle" (OuterVolumeSpecName: "bundle") pod "54d3c703-27b7-486a-b619-001ee40d498f" (UID: "54d3c703-27b7-486a-b619-001ee40d498f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:39:03.080244 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.080212    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54d3c703-27b7-486a-b619-001ee40d498f-kube-api-access-426kk" (OuterVolumeSpecName: "kube-api-access-426kk") pod "54d3c703-27b7-486a-b619-001ee40d498f" (UID: "54d3c703-27b7-486a-b619-001ee40d498f"). InnerVolumeSpecName "kube-api-access-426kk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:39:03.084029 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.084001    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d3c703-27b7-486a-b619-001ee40d498f-util" (OuterVolumeSpecName: "util") pod "54d3c703-27b7-486a-b619-001ee40d498f" (UID: "54d3c703-27b7-486a-b619-001ee40d498f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:39:03.178378 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.178303    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54d3c703-27b7-486a-b619-001ee40d498f-util\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:39:03.178378 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.178332    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-426kk\" (UniqueName: \"kubernetes.io/projected/54d3c703-27b7-486a-b619-001ee40d498f-kube-api-access-426kk\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:39:03.178378 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.178343    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54d3c703-27b7-486a-b619-001ee40d498f-bundle\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:39:03.934100 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.934067    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf"
Sep 14 01:39:03.934100 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.934079    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0pc9kf" event={"ID":"54d3c703-27b7-486a-b619-001ee40d498f","Type":"ContainerDied","Data":"84d8a8d74560e6f6526cb03ba7d8a9b70ad9b59b3bd6ecb5f62e83437e3f035b"}
Sep 14 01:39:03.934553 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.934127    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84d8a8d74560e6f6526cb03ba7d8a9b70ad9b59b3bd6ecb5f62e83437e3f035b"
Sep 14 01:39:03.936155 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.936133    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc" event={"ID":"2674dd1c-d5b3-427c-9386-bc375c7268e7","Type":"ContainerStarted","Data":"33598880fda504b0e56048c5adbacce9f5e55b0c947409a2fef951b1626a6b3a"}
Sep 14 01:39:03.936281 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.936255    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:39:03.959656 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:03.959601    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc" podStartSLOduration=2.533942733 podStartE2EDuration="6.959583044s" totalImagesPullingTime="3.317488484s" totalInitContainerRuntime="1.108151827s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:38:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:39:03.95799948 +0000 UTC m=+503.438450302" watchObservedRunningTime="2026-09-14 01:39:03.959583044 +0000 UTC m=+503.440033864"
Sep 14 01:39:14.943422 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:14.943331    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c5d46b944-lbcsc"
Sep 14 01:39:16.946030 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.945998    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz"]
Sep 14 01:39:16.946518 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.946338    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d3c703-27b7-486a-b619-001ee40d498f" containerName="util"
Sep 14 01:39:16.946518 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.946349    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d3c703-27b7-486a-b619-001ee40d498f" containerName="util"
Sep 14 01:39:16.946518 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.946363    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d3c703-27b7-486a-b619-001ee40d498f" containerName="pull"
Sep 14 01:39:16.946518 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.946369    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d3c703-27b7-486a-b619-001ee40d498f" containerName="pull"
Sep 14 01:39:16.946518 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.946377    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d3c703-27b7-486a-b619-001ee40d498f" containerName="extract"
Sep 14 01:39:16.946518 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.946382    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d3c703-27b7-486a-b619-001ee40d498f" containerName="extract"
Sep 14 01:39:16.946518 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.946444    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="54d3c703-27b7-486a-b619-001ee40d498f" containerName="extract"
Sep 14 01:39:16.955609 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.955585    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz"
Sep 14 01:39:16.957823 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.957794    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 14 01:39:16.957993 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.957847    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 14 01:39:16.958117 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.958102    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-b7s7l\""
Sep 14 01:39:16.965866 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.965838    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz"]
Sep 14 01:39:16.991899 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.991862    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc761bf5-2d66-475f-8c59-3b81cb4d8fb2-tls-certs\") pod \"kube-auth-proxy-7cb7b7df59-zgxzz\" (UID: \"fc761bf5-2d66-475f-8c59-3b81cb4d8fb2\") " pod="openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz"
Sep 14 01:39:16.991899 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.991903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5wbl\" (UniqueName: \"kubernetes.io/projected/fc761bf5-2d66-475f-8c59-3b81cb4d8fb2-kube-api-access-z5wbl\") pod \"kube-auth-proxy-7cb7b7df59-zgxzz\" (UID: \"fc761bf5-2d66-475f-8c59-3b81cb4d8fb2\") " pod="openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz"
Sep 14 01:39:16.992161 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:16.991945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc761bf5-2d66-475f-8c59-3b81cb4d8fb2-tmp\") pod \"kube-auth-proxy-7cb7b7df59-zgxzz\" (UID: \"fc761bf5-2d66-475f-8c59-3b81cb4d8fb2\") " pod="openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz"
Sep 14 01:39:17.092751 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:17.092714    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc761bf5-2d66-475f-8c59-3b81cb4d8fb2-tls-certs\") pod \"kube-auth-proxy-7cb7b7df59-zgxzz\" (UID: \"fc761bf5-2d66-475f-8c59-3b81cb4d8fb2\") " pod="openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz"
Sep 14 01:39:17.092943 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:17.092768    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5wbl\" (UniqueName: \"kubernetes.io/projected/fc761bf5-2d66-475f-8c59-3b81cb4d8fb2-kube-api-access-z5wbl\") pod \"kube-auth-proxy-7cb7b7df59-zgxzz\" (UID: \"fc761bf5-2d66-475f-8c59-3b81cb4d8fb2\") " pod="openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz"
Sep 14 01:39:17.093096 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:17.092945    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc761bf5-2d66-475f-8c59-3b81cb4d8fb2-tmp\") pod \"kube-auth-proxy-7cb7b7df59-zgxzz\" (UID: \"fc761bf5-2d66-475f-8c59-3b81cb4d8fb2\") " pod="openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz"
Sep 14 01:39:17.093756 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:39:17.093728    2581 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 14 01:39:17.093881 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:39:17.093855    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc761bf5-2d66-475f-8c59-3b81cb4d8fb2-tls-certs podName:fc761bf5-2d66-475f-8c59-3b81cb4d8fb2 nodeName:}" failed. No retries permitted until 2026-09-14 01:39:17.593831584 +0000 UTC m=+517.074282404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/fc761bf5-2d66-475f-8c59-3b81cb4d8fb2-tls-certs") pod "kube-auth-proxy-7cb7b7df59-zgxzz" (UID: "fc761bf5-2d66-475f-8c59-3b81cb4d8fb2") : secret "kube-auth-proxy-tls" not found
Sep 14 01:39:17.096244 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:17.096222    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc761bf5-2d66-475f-8c59-3b81cb4d8fb2-tmp\") pod \"kube-auth-proxy-7cb7b7df59-zgxzz\" (UID: \"fc761bf5-2d66-475f-8c59-3b81cb4d8fb2\") " pod="openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz"
Sep 14 01:39:17.100943 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:17.100887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5wbl\" (UniqueName: \"kubernetes.io/projected/fc761bf5-2d66-475f-8c59-3b81cb4d8fb2-kube-api-access-z5wbl\") pod \"kube-auth-proxy-7cb7b7df59-zgxzz\" (UID: \"fc761bf5-2d66-475f-8c59-3b81cb4d8fb2\") " pod="openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz"
Sep 14 01:39:17.598470 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:17.598429    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc761bf5-2d66-475f-8c59-3b81cb4d8fb2-tls-certs\") pod \"kube-auth-proxy-7cb7b7df59-zgxzz\" (UID: \"fc761bf5-2d66-475f-8c59-3b81cb4d8fb2\") " pod="openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz"
Sep 14 01:39:17.600890 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:17.600868    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc761bf5-2d66-475f-8c59-3b81cb4d8fb2-tls-certs\") pod \"kube-auth-proxy-7cb7b7df59-zgxzz\" (UID: \"fc761bf5-2d66-475f-8c59-3b81cb4d8fb2\") " pod="openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz"
Sep 14 01:39:17.866195 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:17.866100    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz"
Sep 14 01:39:18.208555 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.208524    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz"]
Sep 14 01:39:18.211512 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:39:18.211484    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc761bf5_2d66_475f_8c59_3b81cb4d8fb2.slice/crio-d4b5a81bd3dd4888f71e461fefdc416f24701715af0ec320c8ea518513d377bd WatchSource:0}: Error finding container d4b5a81bd3dd4888f71e461fefdc416f24701715af0ec320c8ea518513d377bd: Status 404 returned error can't find the container with id d4b5a81bd3dd4888f71e461fefdc416f24701715af0ec320c8ea518513d377bd
Sep 14 01:39:18.220365 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.220340    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"]
Sep 14 01:39:18.223207 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.223191    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.225403 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.225380    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 14 01:39:18.225403 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.225380    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 14 01:39:18.225568 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.225443    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wbbvm\""
Sep 14 01:39:18.225568 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.225404    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 14 01:39:18.225841 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.225826    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 01:39:18.248333 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.248302    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"]
Sep 14 01:39:18.306180 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.306137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.306377 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.306198    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vswgv\" (UniqueName: \"kubernetes.io/projected/dcbfcc1e-8132-469e-a3cc-afd342747097-kube-api-access-vswgv\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.306377 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.306261    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.306377 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.306324    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.306512 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.306382    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.306512 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.306445    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.306512 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.306471    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcbfcc1e-8132-469e-a3cc-afd342747097-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.407335 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.407293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.407335 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.407338    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcbfcc1e-8132-469e-a3cc-afd342747097-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.407546 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.407356    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.407546 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.407483    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vswgv\" (UniqueName: \"kubernetes.io/projected/dcbfcc1e-8132-469e-a3cc-afd342747097-kube-api-access-vswgv\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.407546 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.407533    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.407684 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.407659    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.407758 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.407739    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.408038 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.408010    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.409837 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.409812    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcbfcc1e-8132-469e-a3cc-afd342747097-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.409980 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.409889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.410035 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.409987    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.410098 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.410078    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.415798 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.415768    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.416068 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.416045    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vswgv\" (UniqueName: \"kubernetes.io/projected/dcbfcc1e-8132-469e-a3cc-afd342747097-kube-api-access-vswgv\") pod \"istiod-openshift-gateway-7b7995d5cd-6ht74\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.534011 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.533888    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:18.808873 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:18.808800    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"]
Sep 14 01:39:18.812227 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:39:18.812193    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcbfcc1e_8132_469e_a3cc_afd342747097.slice/crio-5264c122737e6a1140cf7db2da565b6e6549108e231e6a5f26915d61b0960693 WatchSource:0}: Error finding container 5264c122737e6a1140cf7db2da565b6e6549108e231e6a5f26915d61b0960693: Status 404 returned error can't find the container with id 5264c122737e6a1140cf7db2da565b6e6549108e231e6a5f26915d61b0960693
Sep 14 01:39:19.010802 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:19.010758    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74" event={"ID":"dcbfcc1e-8132-469e-a3cc-afd342747097","Type":"ContainerStarted","Data":"5264c122737e6a1140cf7db2da565b6e6549108e231e6a5f26915d61b0960693"}
Sep 14 01:39:19.012655 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:19.012589    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz" event={"ID":"fc761bf5-2d66-475f-8c59-3b81cb4d8fb2","Type":"ContainerStarted","Data":"d4b5a81bd3dd4888f71e461fefdc416f24701715af0ec320c8ea518513d377bd"}
Sep 14 01:39:21.027050 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:21.026957    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz" event={"ID":"fc761bf5-2d66-475f-8c59-3b81cb4d8fb2","Type":"ContainerStarted","Data":"542d297212a322cb264795929ce3da939b5fe5f0c33ced1807eb42955c660d44"}
Sep 14 01:39:21.043310 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:21.043199    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7cb7b7df59-zgxzz" podStartSLOduration=2.52391851 podStartE2EDuration="5.043175461s" totalImagesPullingTime="2.519256951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:39:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:39:21.040149829 +0000 UTC m=+520.520600652" watchObservedRunningTime="2026-09-14 01:39:21.043175461 +0000 UTC m=+520.523626284"
Sep 14 01:39:22.706217 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:22.706179    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 01:39:22.706463 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:22.706256    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 01:39:23.039516 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:23.039421    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74" event={"ID":"dcbfcc1e-8132-469e-a3cc-afd342747097","Type":"ContainerStarted","Data":"1910597987fd1040b578553bb2ca2d80d4716f7b3e715fc1294162b571f9d643"}
Sep 14 01:39:23.039695 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:23.039624    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:23.041499 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:23.041474    2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-6ht74 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 01:39:23.041616 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:23.041528    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74" podUID="dcbfcc1e-8132-469e-a3cc-afd342747097" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 01:39:23.058186 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:23.058138    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74" podStartSLOduration=1.166896764 podStartE2EDuration="5.058123808s" totalImagesPullingTime="3.891227044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:39:23.05631017 +0000 UTC m=+522.536760993" watchObservedRunningTime="2026-09-14 01:39:23.058123808 +0000 UTC m=+522.538574629"
Sep 14 01:39:24.045987 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:24.045954    2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-6ht74 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 01:39:24.046363 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:24.046005    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74" podUID="dcbfcc1e-8132-469e-a3cc-afd342747097" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 01:39:27.046557 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:27.046525    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:36.398045 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.397959    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"]
Sep 14 01:39:36.401126 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.401109    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:39:36.404319 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.404290    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 14 01:39:36.404319 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.404296    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 14 01:39:36.404541 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.404327    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-68wwc\""
Sep 14 01:39:36.404541 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.404327    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 14 01:39:36.404541 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.404416    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 14 01:39:36.404541 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.404291    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 14 01:39:36.415923 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.415887    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"]
Sep 14 01:39:36.574153 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.574122    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlm9x\" (UniqueName: \"kubernetes.io/projected/513bbf17-afe2-495b-bffa-fc9cb2e23918-kube-api-access-qlm9x\") pod \"lws-controller-manager-5f97d9f44d-4sj9c\" (UID: \"513bbf17-afe2-495b-bffa-fc9cb2e23918\") " pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:39:36.574344 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.574197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/513bbf17-afe2-495b-bffa-fc9cb2e23918-manager-config\") pod \"lws-controller-manager-5f97d9f44d-4sj9c\" (UID: \"513bbf17-afe2-495b-bffa-fc9cb2e23918\") " pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:39:36.574344 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.574223    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/513bbf17-afe2-495b-bffa-fc9cb2e23918-cert\") pod \"lws-controller-manager-5f97d9f44d-4sj9c\" (UID: \"513bbf17-afe2-495b-bffa-fc9cb2e23918\") " pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:39:36.574344 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.574286    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/513bbf17-afe2-495b-bffa-fc9cb2e23918-metrics-cert\") pod \"lws-controller-manager-5f97d9f44d-4sj9c\" (UID: \"513bbf17-afe2-495b-bffa-fc9cb2e23918\") " pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:39:36.675152 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.675058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlm9x\" (UniqueName: \"kubernetes.io/projected/513bbf17-afe2-495b-bffa-fc9cb2e23918-kube-api-access-qlm9x\") pod \"lws-controller-manager-5f97d9f44d-4sj9c\" (UID: \"513bbf17-afe2-495b-bffa-fc9cb2e23918\") " pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:39:36.675152 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.675145    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/513bbf17-afe2-495b-bffa-fc9cb2e23918-manager-config\") pod \"lws-controller-manager-5f97d9f44d-4sj9c\" (UID: \"513bbf17-afe2-495b-bffa-fc9cb2e23918\") " pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:39:36.675383 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.675173    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/513bbf17-afe2-495b-bffa-fc9cb2e23918-cert\") pod \"lws-controller-manager-5f97d9f44d-4sj9c\" (UID: \"513bbf17-afe2-495b-bffa-fc9cb2e23918\") " pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:39:36.675383 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.675314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/513bbf17-afe2-495b-bffa-fc9cb2e23918-metrics-cert\") pod \"lws-controller-manager-5f97d9f44d-4sj9c\" (UID: \"513bbf17-afe2-495b-bffa-fc9cb2e23918\") " pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:39:36.675872 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.675852    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/513bbf17-afe2-495b-bffa-fc9cb2e23918-manager-config\") pod \"lws-controller-manager-5f97d9f44d-4sj9c\" (UID: \"513bbf17-afe2-495b-bffa-fc9cb2e23918\") " pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:39:36.677803 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.677772    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/513bbf17-afe2-495b-bffa-fc9cb2e23918-metrics-cert\") pod \"lws-controller-manager-5f97d9f44d-4sj9c\" (UID: \"513bbf17-afe2-495b-bffa-fc9cb2e23918\") " pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:39:36.678217 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.678198    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/513bbf17-afe2-495b-bffa-fc9cb2e23918-cert\") pod \"lws-controller-manager-5f97d9f44d-4sj9c\" (UID: \"513bbf17-afe2-495b-bffa-fc9cb2e23918\") " pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:39:36.683471 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.683443    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlm9x\" (UniqueName: \"kubernetes.io/projected/513bbf17-afe2-495b-bffa-fc9cb2e23918-kube-api-access-qlm9x\") pod \"lws-controller-manager-5f97d9f44d-4sj9c\" (UID: \"513bbf17-afe2-495b-bffa-fc9cb2e23918\") " pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:39:36.710650 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:36.710614    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:39:37.018088 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:37.018055    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"]
Sep 14 01:39:37.022396 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:39:37.022361    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod513bbf17_afe2_495b_bffa_fc9cb2e23918.slice/crio-52c2386893fba7d4f3d88094c4d8af578fed7a9f0099923d3a1e4e51cf49eb6d WatchSource:0}: Error finding container 52c2386893fba7d4f3d88094c4d8af578fed7a9f0099923d3a1e4e51cf49eb6d: Status 404 returned error can't find the container with id 52c2386893fba7d4f3d88094c4d8af578fed7a9f0099923d3a1e4e51cf49eb6d
Sep 14 01:39:37.112302 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:37.112267    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c" event={"ID":"513bbf17-afe2-495b-bffa-fc9cb2e23918","Type":"ContainerStarted","Data":"52c2386893fba7d4f3d88094c4d8af578fed7a9f0099923d3a1e4e51cf49eb6d"}
Sep 14 01:39:39.124001 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.123889    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c" event={"ID":"513bbf17-afe2-495b-bffa-fc9cb2e23918","Type":"ContainerStarted","Data":"174c8d37b94027adc14df0652f9ba714148c6281dbd0814c65950d44daf603a3"}
Sep 14 01:39:39.124354 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.124050    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:39:39.142662 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.142609    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c" podStartSLOduration=1.3237280120000001 podStartE2EDuration="3.142592687s" totalImagesPullingTime="1.818864675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:39:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:39:39.140784551 +0000 UTC m=+538.621235364" watchObservedRunningTime="2026-09-14 01:39:39.142592687 +0000 UTC m=+538.623043508"
Sep 14 01:39:39.157735 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.157696    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"]
Sep 14 01:39:39.157973 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.157948    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74" podUID="dcbfcc1e-8132-469e-a3cc-afd342747097" containerName="discovery" containerID="cri-o://1910597987fd1040b578553bb2ca2d80d4716f7b3e715fc1294162b571f9d643" gracePeriod=30
Sep 14 01:39:39.422803 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.422779    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:39.602686 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.602655    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-cacerts\") pod \"dcbfcc1e-8132-469e-a3cc-afd342747097\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") "
Sep 14 01:39:39.602686 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.602686    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-token\") pod \"dcbfcc1e-8132-469e-a3cc-afd342747097\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") "
Sep 14 01:39:39.602997 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.602742    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-csr-ca-configmap\") pod \"dcbfcc1e-8132-469e-a3cc-afd342747097\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") "
Sep 14 01:39:39.602997 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.602783    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-csr-dns-cert\") pod \"dcbfcc1e-8132-469e-a3cc-afd342747097\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") "
Sep 14 01:39:39.602997 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.602805    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vswgv\" (UniqueName: \"kubernetes.io/projected/dcbfcc1e-8132-469e-a3cc-afd342747097-kube-api-access-vswgv\") pod \"dcbfcc1e-8132-469e-a3cc-afd342747097\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") "
Sep 14 01:39:39.602997 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.602823    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-kubeconfig\") pod \"dcbfcc1e-8132-469e-a3cc-afd342747097\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") "
Sep 14 01:39:39.602997 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.602866    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcbfcc1e-8132-469e-a3cc-afd342747097-local-certs\") pod \"dcbfcc1e-8132-469e-a3cc-afd342747097\" (UID: \"dcbfcc1e-8132-469e-a3cc-afd342747097\") "
Sep 14 01:39:39.602997 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.603236    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "dcbfcc1e-8132-469e-a3cc-afd342747097" (UID: "dcbfcc1e-8132-469e-a3cc-afd342747097"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:39:39.605216 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.605191    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-token" (OuterVolumeSpecName: "istio-token") pod "dcbfcc1e-8132-469e-a3cc-afd342747097" (UID: "dcbfcc1e-8132-469e-a3cc-afd342747097"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:39:39.605336 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.605192    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-cacerts" (OuterVolumeSpecName: "cacerts") pod "dcbfcc1e-8132-469e-a3cc-afd342747097" (UID: "dcbfcc1e-8132-469e-a3cc-afd342747097"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:39:39.605490 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.605467    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "dcbfcc1e-8132-469e-a3cc-afd342747097" (UID: "dcbfcc1e-8132-469e-a3cc-afd342747097"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:39:39.605583 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.605498    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "dcbfcc1e-8132-469e-a3cc-afd342747097" (UID: "dcbfcc1e-8132-469e-a3cc-afd342747097"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:39:39.605583 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.605523    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcbfcc1e-8132-469e-a3cc-afd342747097-kube-api-access-vswgv" (OuterVolumeSpecName: "kube-api-access-vswgv") pod "dcbfcc1e-8132-469e-a3cc-afd342747097" (UID: "dcbfcc1e-8132-469e-a3cc-afd342747097"). InnerVolumeSpecName "kube-api-access-vswgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:39:39.605583 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.605539    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcbfcc1e-8132-469e-a3cc-afd342747097-local-certs" (OuterVolumeSpecName: "local-certs") pod "dcbfcc1e-8132-469e-a3cc-afd342747097" (UID: "dcbfcc1e-8132-469e-a3cc-afd342747097"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:39:39.703639 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.703616    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-csr-dns-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:39:39.703639 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.703641    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vswgv\" (UniqueName: \"kubernetes.io/projected/dcbfcc1e-8132-469e-a3cc-afd342747097-kube-api-access-vswgv\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:39:39.703842 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.703651    2581 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-kubeconfig\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:39:39.703842 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.703660    2581 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcbfcc1e-8132-469e-a3cc-afd342747097-local-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:39:39.703842 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.703669    2581 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dcbfcc1e-8132-469e-a3cc-afd342747097-cacerts\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:39:39.703842 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.703676    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-token\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:39:39.703842 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:39.703683    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dcbfcc1e-8132-469e-a3cc-afd342747097-istio-csr-ca-configmap\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:39:40.130405 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:40.130311    2581 generic.go:356] "Generic (PLEG): container finished" podID="dcbfcc1e-8132-469e-a3cc-afd342747097" containerID="1910597987fd1040b578553bb2ca2d80d4716f7b3e715fc1294162b571f9d643" exitCode=0
Sep 14 01:39:40.130405 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:40.130360    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"
Sep 14 01:39:40.130881 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:40.130400    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74" event={"ID":"dcbfcc1e-8132-469e-a3cc-afd342747097","Type":"ContainerDied","Data":"1910597987fd1040b578553bb2ca2d80d4716f7b3e715fc1294162b571f9d643"}
Sep 14 01:39:40.130881 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:40.130454    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74" event={"ID":"dcbfcc1e-8132-469e-a3cc-afd342747097","Type":"ContainerDied","Data":"5264c122737e6a1140cf7db2da565b6e6549108e231e6a5f26915d61b0960693"}
Sep 14 01:39:40.130881 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:40.130482    2581 scope.go:117] "RemoveContainer" containerID="1910597987fd1040b578553bb2ca2d80d4716f7b3e715fc1294162b571f9d643"
Sep 14 01:39:40.139239 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:40.139222    2581 scope.go:117] "RemoveContainer" containerID="1910597987fd1040b578553bb2ca2d80d4716f7b3e715fc1294162b571f9d643"
Sep 14 01:39:40.139521 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:39:40.139505    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1910597987fd1040b578553bb2ca2d80d4716f7b3e715fc1294162b571f9d643\": container with ID starting with 1910597987fd1040b578553bb2ca2d80d4716f7b3e715fc1294162b571f9d643 not found: ID does not exist" containerID="1910597987fd1040b578553bb2ca2d80d4716f7b3e715fc1294162b571f9d643"
Sep 14 01:39:40.139566 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:40.139528    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1910597987fd1040b578553bb2ca2d80d4716f7b3e715fc1294162b571f9d643"} err="failed to get container status \"1910597987fd1040b578553bb2ca2d80d4716f7b3e715fc1294162b571f9d643\": rpc error: code = NotFound desc = could not find container \"1910597987fd1040b578553bb2ca2d80d4716f7b3e715fc1294162b571f9d643\": container with ID starting with 1910597987fd1040b578553bb2ca2d80d4716f7b3e715fc1294162b571f9d643 not found: ID does not exist"
Sep 14 01:39:40.150242 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:40.150213    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"]
Sep 14 01:39:40.153984 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:40.153961    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6ht74"]
Sep 14 01:39:41.052969 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:41.052940    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcbfcc1e-8132-469e-a3cc-afd342747097" path="/var/lib/kubelet/pods/dcbfcc1e-8132-469e-a3cc-afd342747097/volumes"
Sep 14 01:39:50.133460 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:39:50.133424    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5f97d9f44d-4sj9c"
Sep 14 01:40:32.284610 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.284578    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"]
Sep 14 01:40:32.285092 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.284950    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcbfcc1e-8132-469e-a3cc-afd342747097" containerName="discovery"
Sep 14 01:40:32.285092 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.284966    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcbfcc1e-8132-469e-a3cc-afd342747097" containerName="discovery"
Sep 14 01:40:32.285092 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.285025    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcbfcc1e-8132-469e-a3cc-afd342747097" containerName="discovery"
Sep 14 01:40:32.287148 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.287132    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"
Sep 14 01:40:32.288926 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.288894    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 01:40:32.289469 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.289447    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tl294\""
Sep 14 01:40:32.289516 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.289501    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 01:40:32.295050 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.295023    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"]
Sep 14 01:40:32.381486 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.381452    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"]
Sep 14 01:40:32.383742 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.383726    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"
Sep 14 01:40:32.392272 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.392242    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"]
Sep 14 01:40:32.455394 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.455355    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/00fd55b1-a119-4659-a18a-c873b7cac7eb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn\" (UID: \"00fd55b1-a119-4659-a18a-c873b7cac7eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"
Sep 14 01:40:32.455394 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.455395    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/00fd55b1-a119-4659-a18a-c873b7cac7eb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn\" (UID: \"00fd55b1-a119-4659-a18a-c873b7cac7eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"
Sep 14 01:40:32.455613 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.455494    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmkls\" (UniqueName: \"kubernetes.io/projected/00fd55b1-a119-4659-a18a-c873b7cac7eb-kube-api-access-wmkls\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn\" (UID: \"00fd55b1-a119-4659-a18a-c873b7cac7eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"
Sep 14 01:40:32.481321 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.481292    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"]
Sep 14 01:40:32.483511 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.483496    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"
Sep 14 01:40:32.492454 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.492432    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"]
Sep 14 01:40:32.556527 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.556431    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm\" (UID: \"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"
Sep 14 01:40:32.556527 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.556486    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfwx4\" (UniqueName: \"kubernetes.io/projected/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-kube-api-access-kfwx4\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm\" (UID: \"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"
Sep 14 01:40:32.556527 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.556525    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmkls\" (UniqueName: \"kubernetes.io/projected/00fd55b1-a119-4659-a18a-c873b7cac7eb-kube-api-access-wmkls\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn\" (UID: \"00fd55b1-a119-4659-a18a-c873b7cac7eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"
Sep 14 01:40:32.556816 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.556555    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/00fd55b1-a119-4659-a18a-c873b7cac7eb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn\" (UID: \"00fd55b1-a119-4659-a18a-c873b7cac7eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"
Sep 14 01:40:32.556816 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.556571    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/00fd55b1-a119-4659-a18a-c873b7cac7eb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn\" (UID: \"00fd55b1-a119-4659-a18a-c873b7cac7eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"
Sep 14 01:40:32.556816 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.556589    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm\" (UID: \"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"
Sep 14 01:40:32.556988 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.556955    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/00fd55b1-a119-4659-a18a-c873b7cac7eb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn\" (UID: \"00fd55b1-a119-4659-a18a-c873b7cac7eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"
Sep 14 01:40:32.557026 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.556989    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/00fd55b1-a119-4659-a18a-c873b7cac7eb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn\" (UID: \"00fd55b1-a119-4659-a18a-c873b7cac7eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"
Sep 14 01:40:32.566953 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.566901    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmkls\" (UniqueName: \"kubernetes.io/projected/00fd55b1-a119-4659-a18a-c873b7cac7eb-kube-api-access-wmkls\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn\" (UID: \"00fd55b1-a119-4659-a18a-c873b7cac7eb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"
Sep 14 01:40:32.583188 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.583153    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"]
Sep 14 01:40:32.585541 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.585525    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"
Sep 14 01:40:32.594485 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.594456    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"]
Sep 14 01:40:32.597830 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.597809    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"
Sep 14 01:40:32.657618 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.657581    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfwx4\" (UniqueName: \"kubernetes.io/projected/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-kube-api-access-kfwx4\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm\" (UID: \"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"
Sep 14 01:40:32.657770 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.657665    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f98a83bf-b97a-44b9-a42f-99df1f209779-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr\" (UID: \"f98a83bf-b97a-44b9-a42f-99df1f209779\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"
Sep 14 01:40:32.657770 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.657701    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm\" (UID: \"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"
Sep 14 01:40:32.657770 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.657760    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f98a83bf-b97a-44b9-a42f-99df1f209779-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr\" (UID: \"f98a83bf-b97a-44b9-a42f-99df1f209779\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"
Sep 14 01:40:32.658396 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.657788    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm\" (UID: \"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"
Sep 14 01:40:32.658396 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.657814    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f59qr\" (UniqueName: \"kubernetes.io/projected/f98a83bf-b97a-44b9-a42f-99df1f209779-kube-api-access-f59qr\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr\" (UID: \"f98a83bf-b97a-44b9-a42f-99df1f209779\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"
Sep 14 01:40:32.658396 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.658269    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm\" (UID: \"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"
Sep 14 01:40:32.658396 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.658289    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm\" (UID: \"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"
Sep 14 01:40:32.666303 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.666272    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfwx4\" (UniqueName: \"kubernetes.io/projected/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-kube-api-access-kfwx4\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm\" (UID: \"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"
Sep 14 01:40:32.694149 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.694118    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"
Sep 14 01:40:32.758516 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.758483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bff05cd-8952-48a1-ae9b-dab28c5545e1-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx\" (UID: \"5bff05cd-8952-48a1-ae9b-dab28c5545e1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"
Sep 14 01:40:32.758682 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.758541    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f98a83bf-b97a-44b9-a42f-99df1f209779-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr\" (UID: \"f98a83bf-b97a-44b9-a42f-99df1f209779\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"
Sep 14 01:40:32.758682 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.758572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f59qr\" (UniqueName: \"kubernetes.io/projected/f98a83bf-b97a-44b9-a42f-99df1f209779-kube-api-access-f59qr\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr\" (UID: \"f98a83bf-b97a-44b9-a42f-99df1f209779\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"
Sep 14 01:40:32.758682 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.758609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75d7q\" (UniqueName: \"kubernetes.io/projected/5bff05cd-8952-48a1-ae9b-dab28c5545e1-kube-api-access-75d7q\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx\" (UID: \"5bff05cd-8952-48a1-ae9b-dab28c5545e1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"
Sep 14 01:40:32.758682 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.758670    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bff05cd-8952-48a1-ae9b-dab28c5545e1-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx\" (UID: \"5bff05cd-8952-48a1-ae9b-dab28c5545e1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"
Sep 14 01:40:32.758902 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.758713    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f98a83bf-b97a-44b9-a42f-99df1f209779-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr\" (UID: \"f98a83bf-b97a-44b9-a42f-99df1f209779\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"
Sep 14 01:40:32.759004 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.758980    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f98a83bf-b97a-44b9-a42f-99df1f209779-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr\" (UID: \"f98a83bf-b97a-44b9-a42f-99df1f209779\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"
Sep 14 01:40:32.759079 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.759058    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f98a83bf-b97a-44b9-a42f-99df1f209779-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr\" (UID: \"f98a83bf-b97a-44b9-a42f-99df1f209779\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"
Sep 14 01:40:32.767600 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.767569    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f59qr\" (UniqueName: \"kubernetes.io/projected/f98a83bf-b97a-44b9-a42f-99df1f209779-kube-api-access-f59qr\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr\" (UID: \"f98a83bf-b97a-44b9-a42f-99df1f209779\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"
Sep 14 01:40:32.793542 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.793507    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"
Sep 14 01:40:32.859688 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.859655    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bff05cd-8952-48a1-ae9b-dab28c5545e1-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx\" (UID: \"5bff05cd-8952-48a1-ae9b-dab28c5545e1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"
Sep 14 01:40:32.859848 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.859721    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bff05cd-8952-48a1-ae9b-dab28c5545e1-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx\" (UID: \"5bff05cd-8952-48a1-ae9b-dab28c5545e1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"
Sep 14 01:40:32.859848 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.859775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75d7q\" (UniqueName: \"kubernetes.io/projected/5bff05cd-8952-48a1-ae9b-dab28c5545e1-kube-api-access-75d7q\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx\" (UID: \"5bff05cd-8952-48a1-ae9b-dab28c5545e1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"
Sep 14 01:40:32.860163 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.860138    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bff05cd-8952-48a1-ae9b-dab28c5545e1-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx\" (UID: \"5bff05cd-8952-48a1-ae9b-dab28c5545e1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"
Sep 14 01:40:32.860278 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.860149    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bff05cd-8952-48a1-ae9b-dab28c5545e1-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx\" (UID: \"5bff05cd-8952-48a1-ae9b-dab28c5545e1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"
Sep 14 01:40:32.867349 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.867319    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75d7q\" (UniqueName: \"kubernetes.io/projected/5bff05cd-8952-48a1-ae9b-dab28c5545e1-kube-api-access-75d7q\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx\" (UID: \"5bff05cd-8952-48a1-ae9b-dab28c5545e1\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"
Sep 14 01:40:32.870214 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.870188    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"]
Sep 14 01:40:32.873074 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:40:32.873044    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00fd55b1_a119_4659_a18a_c873b7cac7eb.slice/crio-439cb2a0c92142153e2e995211df52579a6faed5cac8543e6d8afd6cf6615928 WatchSource:0}: Error finding container 439cb2a0c92142153e2e995211df52579a6faed5cac8543e6d8afd6cf6615928: Status 404 returned error can't find the container with id 439cb2a0c92142153e2e995211df52579a6faed5cac8543e6d8afd6cf6615928
Sep 14 01:40:32.895647 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.895622    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"
Sep 14 01:40:32.967072 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:32.967043    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"]
Sep 14 01:40:32.969252 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:40:32.969222    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1fc1ccf_76c8_4ff9_84b4_5f8ad866a525.slice/crio-c968a78c9022101d1ea7b54feede1ef3f4797b5d614347427f0131593106f6b5 WatchSource:0}: Error finding container c968a78c9022101d1ea7b54feede1ef3f4797b5d614347427f0131593106f6b5: Status 404 returned error can't find the container with id c968a78c9022101d1ea7b54feede1ef3f4797b5d614347427f0131593106f6b5
Sep 14 01:40:33.069151 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:33.069125    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"]
Sep 14 01:40:33.169274 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:33.169254    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"]
Sep 14 01:40:33.174031 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:40:33.174004    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bff05cd_8952_48a1_ae9b_dab28c5545e1.slice/crio-0cc84938acf9feddb9d9580716cf194f74d246be9fabf9ec4e80231a5d364197 WatchSource:0}: Error finding container 0cc84938acf9feddb9d9580716cf194f74d246be9fabf9ec4e80231a5d364197: Status 404 returned error can't find the container with id 0cc84938acf9feddb9d9580716cf194f74d246be9fabf9ec4e80231a5d364197
Sep 14 01:40:33.383882 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:33.383777    2581 generic.go:356] "Generic (PLEG): container finished" podID="00fd55b1-a119-4659-a18a-c873b7cac7eb" containerID="9d5247704d3de78e6a4a8a26268e0538ea4e10b096f198bea47e18d99869d3dc" exitCode=0
Sep 14 01:40:33.384365 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:33.383869    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn" event={"ID":"00fd55b1-a119-4659-a18a-c873b7cac7eb","Type":"ContainerDied","Data":"9d5247704d3de78e6a4a8a26268e0538ea4e10b096f198bea47e18d99869d3dc"}
Sep 14 01:40:33.384365 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:33.383940    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn" event={"ID":"00fd55b1-a119-4659-a18a-c873b7cac7eb","Type":"ContainerStarted","Data":"439cb2a0c92142153e2e995211df52579a6faed5cac8543e6d8afd6cf6615928"}
Sep 14 01:40:33.385641 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:33.385586    2581 generic.go:356] "Generic (PLEG): container finished" podID="d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525" containerID="033e8c656a1dc3f9ff4203050186d82a1efb4683fd2448da96eaed6e1f4a5a07" exitCode=0
Sep 14 01:40:33.385752 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:33.385668    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm" event={"ID":"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525","Type":"ContainerDied","Data":"033e8c656a1dc3f9ff4203050186d82a1efb4683fd2448da96eaed6e1f4a5a07"}
Sep 14 01:40:33.385752 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:33.385721    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm" event={"ID":"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525","Type":"ContainerStarted","Data":"c968a78c9022101d1ea7b54feede1ef3f4797b5d614347427f0131593106f6b5"}
Sep 14 01:40:33.387568 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:33.387549    2581 generic.go:356] "Generic (PLEG): container finished" podID="5bff05cd-8952-48a1-ae9b-dab28c5545e1" containerID="e00f0b31edd0d1898959b44c385d53eee02a18c14050c88f54b4a74d1e51b9e0" exitCode=0
Sep 14 01:40:33.387677 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:33.387616    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx" event={"ID":"5bff05cd-8952-48a1-ae9b-dab28c5545e1","Type":"ContainerDied","Data":"e00f0b31edd0d1898959b44c385d53eee02a18c14050c88f54b4a74d1e51b9e0"}
Sep 14 01:40:33.387677 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:33.387658    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx" event={"ID":"5bff05cd-8952-48a1-ae9b-dab28c5545e1","Type":"ContainerStarted","Data":"0cc84938acf9feddb9d9580716cf194f74d246be9fabf9ec4e80231a5d364197"}
Sep 14 01:40:33.389226 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:33.389205    2581 generic.go:356] "Generic (PLEG): container finished" podID="f98a83bf-b97a-44b9-a42f-99df1f209779" containerID="ce304dd60a4190186681b685b7ec4e523c74146734e40d4e1601d9a2eb2efdb6" exitCode=0
Sep 14 01:40:33.389331 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:33.389264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr" event={"ID":"f98a83bf-b97a-44b9-a42f-99df1f209779","Type":"ContainerDied","Data":"ce304dd60a4190186681b685b7ec4e523c74146734e40d4e1601d9a2eb2efdb6"}
Sep 14 01:40:33.389331 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:33.389296    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr" event={"ID":"f98a83bf-b97a-44b9-a42f-99df1f209779","Type":"ContainerStarted","Data":"d3b7bd18221fccb6bb85cfdc28b3613855f57a59b1e250d44fdc9aae9caa26be"}
Sep 14 01:40:35.400644 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:35.400602    2581 generic.go:356] "Generic (PLEG): container finished" podID="5bff05cd-8952-48a1-ae9b-dab28c5545e1" containerID="62801feeee374c08b4c3e26dff00aaa5e62e424ae22fc063af3427358e0cb772" exitCode=0
Sep 14 01:40:35.401109 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:35.400697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx" event={"ID":"5bff05cd-8952-48a1-ae9b-dab28c5545e1","Type":"ContainerDied","Data":"62801feeee374c08b4c3e26dff00aaa5e62e424ae22fc063af3427358e0cb772"}
Sep 14 01:40:35.402663 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:35.402637    2581 generic.go:356] "Generic (PLEG): container finished" podID="f98a83bf-b97a-44b9-a42f-99df1f209779" containerID="d6d504f78f1d32779d9f5873318061cdca88013eadfa4bf3c4c6704da08b27ff" exitCode=0
Sep 14 01:40:35.402758 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:35.402708    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr" event={"ID":"f98a83bf-b97a-44b9-a42f-99df1f209779","Type":"ContainerDied","Data":"d6d504f78f1d32779d9f5873318061cdca88013eadfa4bf3c4c6704da08b27ff"}
Sep 14 01:40:35.404642 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:35.404622    2581 generic.go:356] "Generic (PLEG): container finished" podID="00fd55b1-a119-4659-a18a-c873b7cac7eb" containerID="c990a1340a655de1e5c7754b140373d533f36b2a065ae8a9029f5cb4e9f1f80a" exitCode=0
Sep 14 01:40:35.404739 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:35.404708    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn" event={"ID":"00fd55b1-a119-4659-a18a-c873b7cac7eb","Type":"ContainerDied","Data":"c990a1340a655de1e5c7754b140373d533f36b2a065ae8a9029f5cb4e9f1f80a"}
Sep 14 01:40:35.406643 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:35.406625    2581 generic.go:356] "Generic (PLEG): container finished" podID="d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525" containerID="401c20ace90e4944e9425bd875e67a26ec0c2a21a6e6e8fa770604529caf54c1" exitCode=0
Sep 14 01:40:35.406725 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:35.406695    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm" event={"ID":"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525","Type":"ContainerDied","Data":"401c20ace90e4944e9425bd875e67a26ec0c2a21a6e6e8fa770604529caf54c1"}
Sep 14 01:40:36.414119 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:36.414073    2581 generic.go:356] "Generic (PLEG): container finished" podID="5bff05cd-8952-48a1-ae9b-dab28c5545e1" containerID="f020e15cb698ac1ec8fc0b51375ebcb7353b52d2a9d9066f5cd0f37453a74ba1" exitCode=0
Sep 14 01:40:36.414592 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:36.414160    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx" event={"ID":"5bff05cd-8952-48a1-ae9b-dab28c5545e1","Type":"ContainerDied","Data":"f020e15cb698ac1ec8fc0b51375ebcb7353b52d2a9d9066f5cd0f37453a74ba1"}
Sep 14 01:40:36.416315 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:36.416292    2581 generic.go:356] "Generic (PLEG): container finished" podID="f98a83bf-b97a-44b9-a42f-99df1f209779" containerID="8c0a785db4223c57fe3addde70d6b23c89e7578228a03f29eb60de76b02da14b" exitCode=0
Sep 14 01:40:36.416421 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:36.416383    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr" event={"ID":"f98a83bf-b97a-44b9-a42f-99df1f209779","Type":"ContainerDied","Data":"8c0a785db4223c57fe3addde70d6b23c89e7578228a03f29eb60de76b02da14b"}
Sep 14 01:40:36.418532 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:36.418512    2581 generic.go:356] "Generic (PLEG): container finished" podID="00fd55b1-a119-4659-a18a-c873b7cac7eb" containerID="bb4c19922616c09fd1f3c974489fb0f6b78e2429ef8a393e9243f3a881bb91ae" exitCode=0
Sep 14 01:40:36.418648 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:36.418598    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn" event={"ID":"00fd55b1-a119-4659-a18a-c873b7cac7eb","Type":"ContainerDied","Data":"bb4c19922616c09fd1f3c974489fb0f6b78e2429ef8a393e9243f3a881bb91ae"}
Sep 14 01:40:36.420639 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:36.420620    2581 generic.go:356] "Generic (PLEG): container finished" podID="d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525" containerID="31340098031b4d1345a7762b9fcdad762a1c5b2c9ba64e7574a2732a442a2d46" exitCode=0
Sep 14 01:40:36.420734 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:36.420662    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm" event={"ID":"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525","Type":"ContainerDied","Data":"31340098031b4d1345a7762b9fcdad762a1c5b2c9ba64e7574a2732a442a2d46"}
Sep 14 01:40:37.625799 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.625773    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"
Sep 14 01:40:37.629323 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.629295    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"
Sep 14 01:40:37.632793 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.632771    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"
Sep 14 01:40:37.636031 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.636011    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"
Sep 14 01:40:37.708119 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.708093    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmkls\" (UniqueName: \"kubernetes.io/projected/00fd55b1-a119-4659-a18a-c873b7cac7eb-kube-api-access-wmkls\") pod \"00fd55b1-a119-4659-a18a-c873b7cac7eb\" (UID: \"00fd55b1-a119-4659-a18a-c873b7cac7eb\") "
Sep 14 01:40:37.708316 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.708124    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f59qr\" (UniqueName: \"kubernetes.io/projected/f98a83bf-b97a-44b9-a42f-99df1f209779-kube-api-access-f59qr\") pod \"f98a83bf-b97a-44b9-a42f-99df1f209779\" (UID: \"f98a83bf-b97a-44b9-a42f-99df1f209779\") "
Sep 14 01:40:37.708316 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.708146    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f98a83bf-b97a-44b9-a42f-99df1f209779-util\") pod \"f98a83bf-b97a-44b9-a42f-99df1f209779\" (UID: \"f98a83bf-b97a-44b9-a42f-99df1f209779\") "
Sep 14 01:40:37.708316 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.708164    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f98a83bf-b97a-44b9-a42f-99df1f209779-bundle\") pod \"f98a83bf-b97a-44b9-a42f-99df1f209779\" (UID: \"f98a83bf-b97a-44b9-a42f-99df1f209779\") "
Sep 14 01:40:37.708316 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.708201    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-util\") pod \"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525\" (UID: \"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525\") "
Sep 14 01:40:37.708316 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.708257    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bff05cd-8952-48a1-ae9b-dab28c5545e1-util\") pod \"5bff05cd-8952-48a1-ae9b-dab28c5545e1\" (UID: \"5bff05cd-8952-48a1-ae9b-dab28c5545e1\") "
Sep 14 01:40:37.708316 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.708295    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/00fd55b1-a119-4659-a18a-c873b7cac7eb-bundle\") pod \"00fd55b1-a119-4659-a18a-c873b7cac7eb\" (UID: \"00fd55b1-a119-4659-a18a-c873b7cac7eb\") "
Sep 14 01:40:37.708639 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.708340    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-bundle\") pod \"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525\" (UID: \"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525\") "
Sep 14 01:40:37.708940 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.708872    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f98a83bf-b97a-44b9-a42f-99df1f209779-bundle" (OuterVolumeSpecName: "bundle") pod "f98a83bf-b97a-44b9-a42f-99df1f209779" (UID: "f98a83bf-b97a-44b9-a42f-99df1f209779"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:40:37.708940 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.708890    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00fd55b1-a119-4659-a18a-c873b7cac7eb-bundle" (OuterVolumeSpecName: "bundle") pod "00fd55b1-a119-4659-a18a-c873b7cac7eb" (UID: "00fd55b1-a119-4659-a18a-c873b7cac7eb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:40:37.709095 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.709064    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfwx4\" (UniqueName: \"kubernetes.io/projected/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-kube-api-access-kfwx4\") pod \"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525\" (UID: \"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525\") "
Sep 14 01:40:37.709315 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.709120    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bff05cd-8952-48a1-ae9b-dab28c5545e1-bundle\") pod \"5bff05cd-8952-48a1-ae9b-dab28c5545e1\" (UID: \"5bff05cd-8952-48a1-ae9b-dab28c5545e1\") "
Sep 14 01:40:37.709373 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.709357    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/00fd55b1-a119-4659-a18a-c873b7cac7eb-util\") pod \"00fd55b1-a119-4659-a18a-c873b7cac7eb\" (UID: \"00fd55b1-a119-4659-a18a-c873b7cac7eb\") "
Sep 14 01:40:37.709430 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.709387    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75d7q\" (UniqueName: \"kubernetes.io/projected/5bff05cd-8952-48a1-ae9b-dab28c5545e1-kube-api-access-75d7q\") pod \"5bff05cd-8952-48a1-ae9b-dab28c5545e1\" (UID: \"5bff05cd-8952-48a1-ae9b-dab28c5545e1\") "
Sep 14 01:40:37.709685 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.709667    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/00fd55b1-a119-4659-a18a-c873b7cac7eb-bundle\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:40:37.709753 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.709691    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f98a83bf-b97a-44b9-a42f-99df1f209779-bundle\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:40:37.709890 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.709867    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-bundle" (OuterVolumeSpecName: "bundle") pod "d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525" (UID: "d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:40:37.710574 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.710550    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bff05cd-8952-48a1-ae9b-dab28c5545e1-bundle" (OuterVolumeSpecName: "bundle") pod "5bff05cd-8952-48a1-ae9b-dab28c5545e1" (UID: "5bff05cd-8952-48a1-ae9b-dab28c5545e1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:40:37.717330 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.711866    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f98a83bf-b97a-44b9-a42f-99df1f209779-kube-api-access-f59qr" (OuterVolumeSpecName: "kube-api-access-f59qr") pod "f98a83bf-b97a-44b9-a42f-99df1f209779" (UID: "f98a83bf-b97a-44b9-a42f-99df1f209779"). InnerVolumeSpecName "kube-api-access-f59qr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:40:37.717330 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.711956    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00fd55b1-a119-4659-a18a-c873b7cac7eb-kube-api-access-wmkls" (OuterVolumeSpecName: "kube-api-access-wmkls") pod "00fd55b1-a119-4659-a18a-c873b7cac7eb" (UID: "00fd55b1-a119-4659-a18a-c873b7cac7eb"). InnerVolumeSpecName "kube-api-access-wmkls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:40:37.717330 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.715169    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bff05cd-8952-48a1-ae9b-dab28c5545e1-util" (OuterVolumeSpecName: "util") pod "5bff05cd-8952-48a1-ae9b-dab28c5545e1" (UID: "5bff05cd-8952-48a1-ae9b-dab28c5545e1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:40:37.717330 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.716959    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-util" (OuterVolumeSpecName: "util") pod "d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525" (UID: "d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:40:37.717538 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.717386    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-kube-api-access-kfwx4" (OuterVolumeSpecName: "kube-api-access-kfwx4") pod "d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525" (UID: "d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525"). InnerVolumeSpecName "kube-api-access-kfwx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:40:37.717538 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.717410    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bff05cd-8952-48a1-ae9b-dab28c5545e1-kube-api-access-75d7q" (OuterVolumeSpecName: "kube-api-access-75d7q") pod "5bff05cd-8952-48a1-ae9b-dab28c5545e1" (UID: "5bff05cd-8952-48a1-ae9b-dab28c5545e1"). InnerVolumeSpecName "kube-api-access-75d7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:40:37.717677 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.717653    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f98a83bf-b97a-44b9-a42f-99df1f209779-util" (OuterVolumeSpecName: "util") pod "f98a83bf-b97a-44b9-a42f-99df1f209779" (UID: "f98a83bf-b97a-44b9-a42f-99df1f209779"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:40:37.718888 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.718871    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00fd55b1-a119-4659-a18a-c873b7cac7eb-util" (OuterVolumeSpecName: "util") pod "00fd55b1-a119-4659-a18a-c873b7cac7eb" (UID: "00fd55b1-a119-4659-a18a-c873b7cac7eb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:40:37.810967 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.810931    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5bff05cd-8952-48a1-ae9b-dab28c5545e1-bundle\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:40:37.810967 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.810960    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/00fd55b1-a119-4659-a18a-c873b7cac7eb-util\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:40:37.810967 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.810969    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75d7q\" (UniqueName: \"kubernetes.io/projected/5bff05cd-8952-48a1-ae9b-dab28c5545e1-kube-api-access-75d7q\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:40:37.810967 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.810979    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmkls\" (UniqueName: \"kubernetes.io/projected/00fd55b1-a119-4659-a18a-c873b7cac7eb-kube-api-access-wmkls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:40:37.811238 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.810989    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f59qr\" (UniqueName: \"kubernetes.io/projected/f98a83bf-b97a-44b9-a42f-99df1f209779-kube-api-access-f59qr\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:40:37.811238 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.810997    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f98a83bf-b97a-44b9-a42f-99df1f209779-util\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:40:37.811238 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.811005    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-util\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:40:37.811238 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.811012    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5bff05cd-8952-48a1-ae9b-dab28c5545e1-util\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:40:37.811238 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.811019    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-bundle\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:40:37.811238 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:37.811027    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfwx4\" (UniqueName: \"kubernetes.io/projected/d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525-kube-api-access-kfwx4\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:40:38.433334 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:38.433298    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx" event={"ID":"5bff05cd-8952-48a1-ae9b-dab28c5545e1","Type":"ContainerDied","Data":"0cc84938acf9feddb9d9580716cf194f74d246be9fabf9ec4e80231a5d364197"}
Sep 14 01:40:38.433334 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:38.433315    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dr6rx"
Sep 14 01:40:38.433571 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:38.433349    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0cc84938acf9feddb9d9580716cf194f74d246be9fabf9ec4e80231a5d364197"
Sep 14 01:40:38.435342 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:38.435322    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr"
Sep 14 01:40:38.435480 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:38.435343    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cn4pqr" event={"ID":"f98a83bf-b97a-44b9-a42f-99df1f209779","Type":"ContainerDied","Data":"d3b7bd18221fccb6bb85cfdc28b3613855f57a59b1e250d44fdc9aae9caa26be"}
Sep 14 01:40:38.435480 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:38.435381    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d3b7bd18221fccb6bb85cfdc28b3613855f57a59b1e250d44fdc9aae9caa26be"
Sep 14 01:40:38.437453 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:38.437434    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn"
Sep 14 01:40:38.437565 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:38.437444    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7msbvn" event={"ID":"00fd55b1-a119-4659-a18a-c873b7cac7eb","Type":"ContainerDied","Data":"439cb2a0c92142153e2e995211df52579a6faed5cac8543e6d8afd6cf6615928"}
Sep 14 01:40:38.437565 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:38.437483    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="439cb2a0c92142153e2e995211df52579a6faed5cac8543e6d8afd6cf6615928"
Sep 14 01:40:38.439696 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:38.439675    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm" event={"ID":"d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525","Type":"ContainerDied","Data":"c968a78c9022101d1ea7b54feede1ef3f4797b5d614347427f0131593106f6b5"}
Sep 14 01:40:38.439793 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:38.439709    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c968a78c9022101d1ea7b54feede1ef3f4797b5d614347427f0131593106f6b5"
Sep 14 01:40:38.439793 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:38.439678    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847armpmm"
Sep 14 01:40:41.014887 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:41.014860    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 01:40:41.015334 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:41.014955    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 01:40:42.315730 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.315694    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79894c44d8-lck7n"]
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316216    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f98a83bf-b97a-44b9-a42f-99df1f209779" containerName="pull"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316233    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f98a83bf-b97a-44b9-a42f-99df1f209779" containerName="pull"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316243    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bff05cd-8952-48a1-ae9b-dab28c5545e1" containerName="extract"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316249    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bff05cd-8952-48a1-ae9b-dab28c5545e1" containerName="extract"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316256    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00fd55b1-a119-4659-a18a-c873b7cac7eb" containerName="extract"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316263    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="00fd55b1-a119-4659-a18a-c873b7cac7eb" containerName="extract"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316270    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f98a83bf-b97a-44b9-a42f-99df1f209779" containerName="util"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316275    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f98a83bf-b97a-44b9-a42f-99df1f209779" containerName="util"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316282    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00fd55b1-a119-4659-a18a-c873b7cac7eb" containerName="util"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316287    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="00fd55b1-a119-4659-a18a-c873b7cac7eb" containerName="util"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316295    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525" containerName="util"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316300    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525" containerName="util"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316305    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bff05cd-8952-48a1-ae9b-dab28c5545e1" containerName="util"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316310    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bff05cd-8952-48a1-ae9b-dab28c5545e1" containerName="util"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316316    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f98a83bf-b97a-44b9-a42f-99df1f209779" containerName="extract"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316321    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f98a83bf-b97a-44b9-a42f-99df1f209779" containerName="extract"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316331    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00fd55b1-a119-4659-a18a-c873b7cac7eb" containerName="pull"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316335    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="00fd55b1-a119-4659-a18a-c873b7cac7eb" containerName="pull"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316345    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bff05cd-8952-48a1-ae9b-dab28c5545e1" containerName="pull"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316352    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bff05cd-8952-48a1-ae9b-dab28c5545e1" containerName="pull"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316361    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525" containerName="pull"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316366    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525" containerName="pull"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316380    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525" containerName="extract"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316385    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525" containerName="extract"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316443    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1fc1ccf-76c8-4ff9-84b4-5f8ad866a525" containerName="extract"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316452    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="00fd55b1-a119-4659-a18a-c873b7cac7eb" containerName="extract"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316457    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f98a83bf-b97a-44b9-a42f-99df1f209779" containerName="extract"
Sep 14 01:40:42.318090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.316463    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bff05cd-8952-48a1-ae9b-dab28c5545e1" containerName="extract"
Sep 14 01:40:42.319246 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.319224    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.338353 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.338326    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79894c44d8-lck7n"]
Sep 14 01:40:42.351406 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.351365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-service-ca\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.351665 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.351644    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-console-config\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.351831 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.351810    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlfbp\" (UniqueName: \"kubernetes.io/projected/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-kube-api-access-nlfbp\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.352045 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.352019    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-console-serving-cert\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.352148 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.352061    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-oauth-serving-cert\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.352148 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.352089    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-console-oauth-config\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.352280 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.352159    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-trusted-ca-bundle\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.453289 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.453232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-service-ca\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.453289 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.453288    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-console-config\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.453524 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.453402    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlfbp\" (UniqueName: \"kubernetes.io/projected/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-kube-api-access-nlfbp\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.453524 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.453465    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-console-serving-cert\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.453661 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.453639    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-oauth-serving-cert\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.453725 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.453695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-console-oauth-config\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.453775 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.453741    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-trusted-ca-bundle\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.454030 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.454003    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-console-config\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.454160 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.454075    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-service-ca\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.454297 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.454275    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-oauth-serving-cert\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.454517 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.454497    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-trusted-ca-bundle\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.455947 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.455930    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-console-serving-cert\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.456033 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.456016    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-console-oauth-config\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.465492 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.465468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlfbp\" (UniqueName: \"kubernetes.io/projected/1a154ab1-6eb3-4772-810c-8beb7b4ee52c-kube-api-access-nlfbp\") pod \"console-79894c44d8-lck7n\" (UID: \"1a154ab1-6eb3-4772-810c-8beb7b4ee52c\") " pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.628617 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.628524    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:42.905980 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:42.905955    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79894c44d8-lck7n"]
Sep 14 01:40:42.907832 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:40:42.907802    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a154ab1_6eb3_4772_810c_8beb7b4ee52c.slice/crio-7846c2a5f369ac9aab5e5e8980b66363e9c7f88b77e67efc0e71bd283e98b3d7 WatchSource:0}: Error finding container 7846c2a5f369ac9aab5e5e8980b66363e9c7f88b77e67efc0e71bd283e98b3d7: Status 404 returned error can't find the container with id 7846c2a5f369ac9aab5e5e8980b66363e9c7f88b77e67efc0e71bd283e98b3d7
Sep 14 01:40:43.470587 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:43.470553    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79894c44d8-lck7n" event={"ID":"1a154ab1-6eb3-4772-810c-8beb7b4ee52c","Type":"ContainerStarted","Data":"06a749ac5698cc810b17268817043061bf96449ecae66edde0b563e5cf5861d4"}
Sep 14 01:40:43.471009 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:43.470602    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79894c44d8-lck7n" event={"ID":"1a154ab1-6eb3-4772-810c-8beb7b4ee52c","Type":"ContainerStarted","Data":"7846c2a5f369ac9aab5e5e8980b66363e9c7f88b77e67efc0e71bd283e98b3d7"}
Sep 14 01:40:43.486583 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:43.486524    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79894c44d8-lck7n" podStartSLOduration=1.4865032519999999 podStartE2EDuration="1.486503252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:40:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:40:43.486052943 +0000 UTC m=+602.966503765" watchObservedRunningTime="2026-09-14 01:40:43.486503252 +0000 UTC m=+602.966954075"
Sep 14 01:40:52.629086 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:52.629048    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:52.629531 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:52.629149    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:52.633962 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:52.633938    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:53.525104 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:53.525073    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79894c44d8-lck7n"
Sep 14 01:40:53.577041 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:53.577005    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-658dd4fdff-4cx6n"]
Sep 14 01:40:53.979347 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:53.979313    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf"]
Sep 14 01:40:53.982876 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:53.982858    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf"
Sep 14 01:40:53.985070 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:53.985047    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 14 01:40:53.985247 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:53.985100    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2nlnr\""
Sep 14 01:40:53.985247 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:53.985109    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 14 01:40:53.985348 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:53.985260    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 14 01:40:53.985348 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:53.985266    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 14 01:40:53.990077 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:53.990050    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf"]
Sep 14 01:40:54.058398 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:54.058363    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4c2de50f-539a-434c-acd0-a7ed170f55e9-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-q2lpf\" (UID: \"4c2de50f-539a-434c-acd0-a7ed170f55e9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf"
Sep 14 01:40:54.058398 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:54.058402    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c2de50f-539a-434c-acd0-a7ed170f55e9-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-q2lpf\" (UID: \"4c2de50f-539a-434c-acd0-a7ed170f55e9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf"
Sep 14 01:40:54.058662 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:54.058514    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt7ld\" (UniqueName: \"kubernetes.io/projected/4c2de50f-539a-434c-acd0-a7ed170f55e9-kube-api-access-rt7ld\") pod \"kuadrant-console-plugin-fbdd4f445-q2lpf\" (UID: \"4c2de50f-539a-434c-acd0-a7ed170f55e9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf"
Sep 14 01:40:54.160002 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:54.159878    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt7ld\" (UniqueName: \"kubernetes.io/projected/4c2de50f-539a-434c-acd0-a7ed170f55e9-kube-api-access-rt7ld\") pod \"kuadrant-console-plugin-fbdd4f445-q2lpf\" (UID: \"4c2de50f-539a-434c-acd0-a7ed170f55e9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf"
Sep 14 01:40:54.160225 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:54.160046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4c2de50f-539a-434c-acd0-a7ed170f55e9-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-q2lpf\" (UID: \"4c2de50f-539a-434c-acd0-a7ed170f55e9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf"
Sep 14 01:40:54.160225 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:54.160079    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c2de50f-539a-434c-acd0-a7ed170f55e9-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-q2lpf\" (UID: \"4c2de50f-539a-434c-acd0-a7ed170f55e9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf"
Sep 14 01:40:54.160341 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:40:54.160252    2581 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 14 01:40:54.160341 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:40:54.160328    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c2de50f-539a-434c-acd0-a7ed170f55e9-plugin-serving-cert podName:4c2de50f-539a-434c-acd0-a7ed170f55e9 nodeName:}" failed. No retries permitted until 2026-09-14 01:40:54.660310575 +0000 UTC m=+614.140761386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/4c2de50f-539a-434c-acd0-a7ed170f55e9-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-q2lpf" (UID: "4c2de50f-539a-434c-acd0-a7ed170f55e9") : secret "plugin-serving-cert" not found
Sep 14 01:40:54.160771 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:54.160746    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4c2de50f-539a-434c-acd0-a7ed170f55e9-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-q2lpf\" (UID: \"4c2de50f-539a-434c-acd0-a7ed170f55e9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf"
Sep 14 01:40:54.168502 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:54.168479    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt7ld\" (UniqueName: \"kubernetes.io/projected/4c2de50f-539a-434c-acd0-a7ed170f55e9-kube-api-access-rt7ld\") pod \"kuadrant-console-plugin-fbdd4f445-q2lpf\" (UID: \"4c2de50f-539a-434c-acd0-a7ed170f55e9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf"
Sep 14 01:40:54.664935 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:54.664872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c2de50f-539a-434c-acd0-a7ed170f55e9-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-q2lpf\" (UID: \"4c2de50f-539a-434c-acd0-a7ed170f55e9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf"
Sep 14 01:40:54.667562 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:54.667524    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c2de50f-539a-434c-acd0-a7ed170f55e9-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-q2lpf\" (UID: \"4c2de50f-539a-434c-acd0-a7ed170f55e9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf"
Sep 14 01:40:54.893785 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:54.893735    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf"
Sep 14 01:40:55.163028 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:55.163000    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf"]
Sep 14 01:40:55.165336 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:40:55.165309    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c2de50f_539a_434c_acd0_a7ed170f55e9.slice/crio-9e3f775419ccba25ac9c286401ff5fbd3e3d3468e2263193d12e6990b006a8aa WatchSource:0}: Error finding container 9e3f775419ccba25ac9c286401ff5fbd3e3d3468e2263193d12e6990b006a8aa: Status 404 returned error can't find the container with id 9e3f775419ccba25ac9c286401ff5fbd3e3d3468e2263193d12e6990b006a8aa
Sep 14 01:40:55.166616 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:55.166597    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 01:40:55.531957 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:40:55.531924    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf" event={"ID":"4c2de50f-539a-434c-acd0-a7ed170f55e9","Type":"ContainerStarted","Data":"9e3f775419ccba25ac9c286401ff5fbd3e3d3468e2263193d12e6990b006a8aa"}
Sep 14 01:41:00.565520 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:00.565483    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf" event={"ID":"4c2de50f-539a-434c-acd0-a7ed170f55e9","Type":"ContainerStarted","Data":"15e0bc3536e4aaef5d6ce8d1cfd14d7c76308572065c47ce2ae4d1d8063778f2"}
Sep 14 01:41:00.580579 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:00.580526    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2lpf" podStartSLOduration=2.32257557 podStartE2EDuration="7.580511496s" totalImagesPullingTime="5.257935926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:41:00.579000219 +0000 UTC m=+620.059451041" watchObservedRunningTime="2026-09-14 01:41:00.580511496 +0000 UTC m=+620.060962317"
Sep 14 01:41:18.601077 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.601031    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-658dd4fdff-4cx6n" podUID="16fcc77f-8774-43a4-bea9-e85ee7b46998" containerName="console" containerID="cri-o://af23404858d3f61a3197aa8e924e575bb4917e264d8d4ec77de57de9e0bed5a7" gracePeriod=15
Sep 14 01:41:18.848766 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.848742    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:41:18.976520 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.976487    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-oauth-serving-cert\") pod \"16fcc77f-8774-43a4-bea9-e85ee7b46998\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") "
Sep 14 01:41:18.976687 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.976534    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-serving-cert\") pod \"16fcc77f-8774-43a4-bea9-e85ee7b46998\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") "
Sep 14 01:41:18.976687 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.976564    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-service-ca\") pod \"16fcc77f-8774-43a4-bea9-e85ee7b46998\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") "
Sep 14 01:41:18.976687 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.976593    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-trusted-ca-bundle\") pod \"16fcc77f-8774-43a4-bea9-e85ee7b46998\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") "
Sep 14 01:41:18.976841 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.976773    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-config\") pod \"16fcc77f-8774-43a4-bea9-e85ee7b46998\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") "
Sep 14 01:41:18.976841 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.976821    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42jmf\" (UniqueName: \"kubernetes.io/projected/16fcc77f-8774-43a4-bea9-e85ee7b46998-kube-api-access-42jmf\") pod \"16fcc77f-8774-43a4-bea9-e85ee7b46998\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") "
Sep 14 01:41:18.976936 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.976896    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-oauth-config\") pod \"16fcc77f-8774-43a4-bea9-e85ee7b46998\" (UID: \"16fcc77f-8774-43a4-bea9-e85ee7b46998\") "
Sep 14 01:41:18.977063 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.977035    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-service-ca" (OuterVolumeSpecName: "service-ca") pod "16fcc77f-8774-43a4-bea9-e85ee7b46998" (UID: "16fcc77f-8774-43a4-bea9-e85ee7b46998"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:41:18.977155 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.977059    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "16fcc77f-8774-43a4-bea9-e85ee7b46998" (UID: "16fcc77f-8774-43a4-bea9-e85ee7b46998"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:41:18.977155 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.977081    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "16fcc77f-8774-43a4-bea9-e85ee7b46998" (UID: "16fcc77f-8774-43a4-bea9-e85ee7b46998"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:41:18.977259 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.977146    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-config" (OuterVolumeSpecName: "console-config") pod "16fcc77f-8774-43a4-bea9-e85ee7b46998" (UID: "16fcc77f-8774-43a4-bea9-e85ee7b46998"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:41:18.977307 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.977271    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-oauth-serving-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:41:18.977307 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.977289    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:41:18.977307 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.977298    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-trusted-ca-bundle\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:41:18.977307 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.977306    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-config\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:41:18.978815 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.978791    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "16fcc77f-8774-43a4-bea9-e85ee7b46998" (UID: "16fcc77f-8774-43a4-bea9-e85ee7b46998"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:41:18.978936 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.978879    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16fcc77f-8774-43a4-bea9-e85ee7b46998-kube-api-access-42jmf" (OuterVolumeSpecName: "kube-api-access-42jmf") pod "16fcc77f-8774-43a4-bea9-e85ee7b46998" (UID: "16fcc77f-8774-43a4-bea9-e85ee7b46998"). InnerVolumeSpecName "kube-api-access-42jmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:41:18.979005 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:18.978965    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "16fcc77f-8774-43a4-bea9-e85ee7b46998" (UID: "16fcc77f-8774-43a4-bea9-e85ee7b46998"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:41:19.078143 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:19.078116    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-serving-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:41:19.078143 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:19.078140    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42jmf\" (UniqueName: \"kubernetes.io/projected/16fcc77f-8774-43a4-bea9-e85ee7b46998-kube-api-access-42jmf\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:41:19.078143 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:19.078149    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16fcc77f-8774-43a4-bea9-e85ee7b46998-console-oauth-config\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:41:19.673687 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:19.673650    2581 generic.go:356] "Generic (PLEG): container finished" podID="16fcc77f-8774-43a4-bea9-e85ee7b46998" containerID="af23404858d3f61a3197aa8e924e575bb4917e264d8d4ec77de57de9e0bed5a7" exitCode=0
Sep 14 01:41:19.674139 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:19.673710    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658dd4fdff-4cx6n"
Sep 14 01:41:19.674139 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:19.673723    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658dd4fdff-4cx6n" event={"ID":"16fcc77f-8774-43a4-bea9-e85ee7b46998","Type":"ContainerDied","Data":"af23404858d3f61a3197aa8e924e575bb4917e264d8d4ec77de57de9e0bed5a7"}
Sep 14 01:41:19.674139 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:19.673792    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658dd4fdff-4cx6n" event={"ID":"16fcc77f-8774-43a4-bea9-e85ee7b46998","Type":"ContainerDied","Data":"77f295d095b6cca05d8d97e3e963cffbaf07e516114e9827e6de7a2a80b100f7"}
Sep 14 01:41:19.674139 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:19.673813    2581 scope.go:117] "RemoveContainer" containerID="af23404858d3f61a3197aa8e924e575bb4917e264d8d4ec77de57de9e0bed5a7"
Sep 14 01:41:19.682627 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:19.682608    2581 scope.go:117] "RemoveContainer" containerID="af23404858d3f61a3197aa8e924e575bb4917e264d8d4ec77de57de9e0bed5a7"
Sep 14 01:41:19.682925 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:41:19.682893    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af23404858d3f61a3197aa8e924e575bb4917e264d8d4ec77de57de9e0bed5a7\": container with ID starting with af23404858d3f61a3197aa8e924e575bb4917e264d8d4ec77de57de9e0bed5a7 not found: ID does not exist" containerID="af23404858d3f61a3197aa8e924e575bb4917e264d8d4ec77de57de9e0bed5a7"
Sep 14 01:41:19.683002 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:19.682939    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af23404858d3f61a3197aa8e924e575bb4917e264d8d4ec77de57de9e0bed5a7"} err="failed to get container status \"af23404858d3f61a3197aa8e924e575bb4917e264d8d4ec77de57de9e0bed5a7\": rpc error: code = NotFound desc = could not find container \"af23404858d3f61a3197aa8e924e575bb4917e264d8d4ec77de57de9e0bed5a7\": container with ID starting with af23404858d3f61a3197aa8e924e575bb4917e264d8d4ec77de57de9e0bed5a7 not found: ID does not exist"
Sep 14 01:41:19.690137 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:19.690112    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-658dd4fdff-4cx6n"]
Sep 14 01:41:19.696746 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:19.696714    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-658dd4fdff-4cx6n"]
Sep 14 01:41:21.053558 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:21.053525    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16fcc77f-8774-43a4-bea9-e85ee7b46998" path="/var/lib/kubelet/pods/16fcc77f-8774-43a4-bea9-e85ee7b46998/volumes"
Sep 14 01:41:46.536561 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:46.536525    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:41:46.537002 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:46.536862    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16fcc77f-8774-43a4-bea9-e85ee7b46998" containerName="console"
Sep 14 01:41:46.537002 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:46.536873    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="16fcc77f-8774-43a4-bea9-e85ee7b46998" containerName="console"
Sep 14 01:41:46.537002 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:46.536958    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="16fcc77f-8774-43a4-bea9-e85ee7b46998" containerName="console"
Sep 14 01:41:46.541110 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:46.541092    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"
Sep 14 01:41:46.543606 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:46.543577    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 14 01:41:46.547467 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:46.547443    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:41:46.570485 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:46.570449    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:41:46.611591 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:46.611556    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58-config-file\") pod \"limitador-limitador-84994f7ffc-tm7z5\" (UID: \"ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"
Sep 14 01:41:46.611747 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:46.611623    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88zrd\" (UniqueName: \"kubernetes.io/projected/ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58-kube-api-access-88zrd\") pod \"limitador-limitador-84994f7ffc-tm7z5\" (UID: \"ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"
Sep 14 01:41:46.712616 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:46.712575    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58-config-file\") pod \"limitador-limitador-84994f7ffc-tm7z5\" (UID: \"ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"
Sep 14 01:41:46.712816 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:46.712652    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88zrd\" (UniqueName: \"kubernetes.io/projected/ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58-kube-api-access-88zrd\") pod \"limitador-limitador-84994f7ffc-tm7z5\" (UID: \"ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"
Sep 14 01:41:46.713275 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:46.713254    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58-config-file\") pod \"limitador-limitador-84994f7ffc-tm7z5\" (UID: \"ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"
Sep 14 01:41:46.726083 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:46.726056    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88zrd\" (UniqueName: \"kubernetes.io/projected/ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58-kube-api-access-88zrd\") pod \"limitador-limitador-84994f7ffc-tm7z5\" (UID: \"ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"
Sep 14 01:41:46.852157 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:46.852067    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"
Sep 14 01:41:47.080167 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:47.080131    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-rqmn5"]
Sep 14 01:41:47.083830 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:47.083806    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-rqmn5"
Sep 14 01:41:47.085939 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:47.085898    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gr8lh\""
Sep 14 01:41:47.090232 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:47.090208    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-rqmn5"]
Sep 14 01:41:47.126096 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:47.126070    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:41:47.128436 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:41:47.128409    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac8c2ae0_32cb_4a1c_b516_728b5fcc0d58.slice/crio-8ac524f82644929f6b299951d3ca523ff1a2d201b9be6091f848fb6ba14e58bd WatchSource:0}: Error finding container 8ac524f82644929f6b299951d3ca523ff1a2d201b9be6091f848fb6ba14e58bd: Status 404 returned error can't find the container with id 8ac524f82644929f6b299951d3ca523ff1a2d201b9be6091f848fb6ba14e58bd
Sep 14 01:41:47.216831 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:47.216795    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw5tc\" (UniqueName: \"kubernetes.io/projected/a119049f-ebf8-4ee8-8708-ee9c54509dcb-kube-api-access-zw5tc\") pod \"authorino-7bcfd8f9cd-rqmn5\" (UID: \"a119049f-ebf8-4ee8-8708-ee9c54509dcb\") " pod="kuadrant-system/authorino-7bcfd8f9cd-rqmn5"
Sep 14 01:41:47.317531 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:47.317490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw5tc\" (UniqueName: \"kubernetes.io/projected/a119049f-ebf8-4ee8-8708-ee9c54509dcb-kube-api-access-zw5tc\") pod \"authorino-7bcfd8f9cd-rqmn5\" (UID: \"a119049f-ebf8-4ee8-8708-ee9c54509dcb\") " pod="kuadrant-system/authorino-7bcfd8f9cd-rqmn5"
Sep 14 01:41:47.325541 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:47.325509    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw5tc\" (UniqueName: \"kubernetes.io/projected/a119049f-ebf8-4ee8-8708-ee9c54509dcb-kube-api-access-zw5tc\") pod \"authorino-7bcfd8f9cd-rqmn5\" (UID: \"a119049f-ebf8-4ee8-8708-ee9c54509dcb\") " pod="kuadrant-system/authorino-7bcfd8f9cd-rqmn5"
Sep 14 01:41:47.394658 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:47.394554    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-rqmn5"
Sep 14 01:41:47.664008 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:47.663931    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-rqmn5"]
Sep 14 01:41:47.667292 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:41:47.667266    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda119049f_ebf8_4ee8_8708_ee9c54509dcb.slice/crio-4715153cbff9cee5c8dae634a4652c6ff3d4e09c14fccf03a72f252af1590923 WatchSource:0}: Error finding container 4715153cbff9cee5c8dae634a4652c6ff3d4e09c14fccf03a72f252af1590923: Status 404 returned error can't find the container with id 4715153cbff9cee5c8dae634a4652c6ff3d4e09c14fccf03a72f252af1590923
Sep 14 01:41:47.818406 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:47.818364    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-rqmn5" event={"ID":"a119049f-ebf8-4ee8-8708-ee9c54509dcb","Type":"ContainerStarted","Data":"4715153cbff9cee5c8dae634a4652c6ff3d4e09c14fccf03a72f252af1590923"}
Sep 14 01:41:47.819654 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:47.819633    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-tm7z5" event={"ID":"ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58","Type":"ContainerStarted","Data":"8ac524f82644929f6b299951d3ca523ff1a2d201b9be6091f848fb6ba14e58bd"}
Sep 14 01:41:50.753702 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:50.753666    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-rqmn5"]
Sep 14 01:41:50.840776 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:50.840738    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-rqmn5" event={"ID":"a119049f-ebf8-4ee8-8708-ee9c54509dcb","Type":"ContainerStarted","Data":"17caea0a17b6e6b3cbf154d91a0105a854d84ff05444e3db2d10420f67478f32"}
Sep 14 01:41:50.842352 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:50.842327    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-tm7z5" event={"ID":"ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58","Type":"ContainerStarted","Data":"a55131c26db6b09d47d76ef8e0852834711e23906d05f1a4d8d46b68e0ec8b9f"}
Sep 14 01:41:50.842490 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:50.842388    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"
Sep 14 01:41:50.853779 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:50.853724    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-rqmn5" podStartSLOduration=1.110806214 podStartE2EDuration="3.853705187s" totalImagesPullingTime="2.742898973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:41:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:41:50.853105777 +0000 UTC m=+670.333556598" watchObservedRunningTime="2026-09-14 01:41:50.853705187 +0000 UTC m=+670.334156010"
Sep 14 01:41:50.868876 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:50.868824    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-tm7z5" podStartSLOduration=1.59380146 podStartE2EDuration="4.868807352s" totalImagesPullingTime="3.275005892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:41:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:41:50.867082481 +0000 UTC m=+670.347533303" watchObservedRunningTime="2026-09-14 01:41:50.868807352 +0000 UTC m=+670.349258173"
Sep 14 01:41:51.847448 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:51.847412    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-rqmn5" podUID="a119049f-ebf8-4ee8-8708-ee9c54509dcb" containerName="authorino" containerID="cri-o://17caea0a17b6e6b3cbf154d91a0105a854d84ff05444e3db2d10420f67478f32" gracePeriod=30
Sep 14 01:41:52.105629 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:52.105561    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-rqmn5"
Sep 14 01:41:52.261591 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:52.261557    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zw5tc\" (UniqueName: \"kubernetes.io/projected/a119049f-ebf8-4ee8-8708-ee9c54509dcb-kube-api-access-zw5tc\") pod \"a119049f-ebf8-4ee8-8708-ee9c54509dcb\" (UID: \"a119049f-ebf8-4ee8-8708-ee9c54509dcb\") "
Sep 14 01:41:52.263900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:52.263877    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a119049f-ebf8-4ee8-8708-ee9c54509dcb-kube-api-access-zw5tc" (OuterVolumeSpecName: "kube-api-access-zw5tc") pod "a119049f-ebf8-4ee8-8708-ee9c54509dcb" (UID: "a119049f-ebf8-4ee8-8708-ee9c54509dcb"). InnerVolumeSpecName "kube-api-access-zw5tc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:41:52.362837 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:52.362752    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zw5tc\" (UniqueName: \"kubernetes.io/projected/a119049f-ebf8-4ee8-8708-ee9c54509dcb-kube-api-access-zw5tc\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:41:52.853253 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:52.853214    2581 generic.go:356] "Generic (PLEG): container finished" podID="a119049f-ebf8-4ee8-8708-ee9c54509dcb" containerID="17caea0a17b6e6b3cbf154d91a0105a854d84ff05444e3db2d10420f67478f32" exitCode=0
Sep 14 01:41:52.853672 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:52.853259    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-rqmn5"
Sep 14 01:41:52.853672 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:52.853296    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-rqmn5" event={"ID":"a119049f-ebf8-4ee8-8708-ee9c54509dcb","Type":"ContainerDied","Data":"17caea0a17b6e6b3cbf154d91a0105a854d84ff05444e3db2d10420f67478f32"}
Sep 14 01:41:52.853672 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:52.853344    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-rqmn5" event={"ID":"a119049f-ebf8-4ee8-8708-ee9c54509dcb","Type":"ContainerDied","Data":"4715153cbff9cee5c8dae634a4652c6ff3d4e09c14fccf03a72f252af1590923"}
Sep 14 01:41:52.853672 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:52.853361    2581 scope.go:117] "RemoveContainer" containerID="17caea0a17b6e6b3cbf154d91a0105a854d84ff05444e3db2d10420f67478f32"
Sep 14 01:41:52.862784 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:52.862766    2581 scope.go:117] "RemoveContainer" containerID="17caea0a17b6e6b3cbf154d91a0105a854d84ff05444e3db2d10420f67478f32"
Sep 14 01:41:52.863087 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:41:52.863069    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17caea0a17b6e6b3cbf154d91a0105a854d84ff05444e3db2d10420f67478f32\": container with ID starting with 17caea0a17b6e6b3cbf154d91a0105a854d84ff05444e3db2d10420f67478f32 not found: ID does not exist" containerID="17caea0a17b6e6b3cbf154d91a0105a854d84ff05444e3db2d10420f67478f32"
Sep 14 01:41:52.863141 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:52.863095    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17caea0a17b6e6b3cbf154d91a0105a854d84ff05444e3db2d10420f67478f32"} err="failed to get container status \"17caea0a17b6e6b3cbf154d91a0105a854d84ff05444e3db2d10420f67478f32\": rpc error: code = NotFound desc = could not find container \"17caea0a17b6e6b3cbf154d91a0105a854d84ff05444e3db2d10420f67478f32\": container with ID starting with 17caea0a17b6e6b3cbf154d91a0105a854d84ff05444e3db2d10420f67478f32 not found: ID does not exist"
Sep 14 01:41:52.873035 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:52.873009    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-rqmn5"]
Sep 14 01:41:52.876182 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:52.876161    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-rqmn5"]
Sep 14 01:41:53.053449 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:41:53.053416    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a119049f-ebf8-4ee8-8708-ee9c54509dcb" path="/var/lib/kubelet/pods/a119049f-ebf8-4ee8-8708-ee9c54509dcb/volumes"
Sep 14 01:42:01.849010 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:01.848978    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"
Sep 14 01:42:16.050265 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.050226    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-v5pt4"]
Sep 14 01:42:16.050804 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.050588    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a119049f-ebf8-4ee8-8708-ee9c54509dcb" containerName="authorino"
Sep 14 01:42:16.050804 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.050600    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a119049f-ebf8-4ee8-8708-ee9c54509dcb" containerName="authorino"
Sep 14 01:42:16.050804 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.050655    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a119049f-ebf8-4ee8-8708-ee9c54509dcb" containerName="authorino"
Sep 14 01:42:16.053369 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.053348    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-v5pt4"
Sep 14 01:42:16.055360 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.055339    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gr8lh\""
Sep 14 01:42:16.059551 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.059527    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-v5pt4"]
Sep 14 01:42:16.070333 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.070305    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfmlg\" (UniqueName: \"kubernetes.io/projected/616f7462-efb8-43e6-858c-d294cf2e59bd-kube-api-access-sfmlg\") pod \"authorino-557f876f69-v5pt4\" (UID: \"616f7462-efb8-43e6-858c-d294cf2e59bd\") " pod="kuadrant-system/authorino-557f876f69-v5pt4"
Sep 14 01:42:16.171057 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.171020    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfmlg\" (UniqueName: \"kubernetes.io/projected/616f7462-efb8-43e6-858c-d294cf2e59bd-kube-api-access-sfmlg\") pod \"authorino-557f876f69-v5pt4\" (UID: \"616f7462-efb8-43e6-858c-d294cf2e59bd\") " pod="kuadrant-system/authorino-557f876f69-v5pt4"
Sep 14 01:42:16.177601 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.177571    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-v5pt4"]
Sep 14 01:42:16.177856 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:42:16.177838    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-sfmlg], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-v5pt4" podUID="616f7462-efb8-43e6-858c-d294cf2e59bd"
Sep 14 01:42:16.179759 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.179739    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfmlg\" (UniqueName: \"kubernetes.io/projected/616f7462-efb8-43e6-858c-d294cf2e59bd-kube-api-access-sfmlg\") pod \"authorino-557f876f69-v5pt4\" (UID: \"616f7462-efb8-43e6-858c-d294cf2e59bd\") " pod="kuadrant-system/authorino-557f876f69-v5pt4"
Sep 14 01:42:16.200552 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.200521    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-wq8h4"]
Sep 14 01:42:16.204075 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.204054    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-wq8h4"
Sep 14 01:42:16.206016 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.205998    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 14 01:42:16.209289 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.209269    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-wq8h4"]
Sep 14 01:42:16.270417 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.270380    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-wq8h4"]
Sep 14 01:42:16.270715 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:42:16.270696    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-wxgwl tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-wq8h4" podUID="48a8315b-780b-448f-8ce5-e57d179cdb48"
Sep 14 01:42:16.272332 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.272305    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxgwl\" (UniqueName: \"kubernetes.io/projected/48a8315b-780b-448f-8ce5-e57d179cdb48-kube-api-access-wxgwl\") pod \"authorino-6d4dfc649f-wq8h4\" (UID: \"48a8315b-780b-448f-8ce5-e57d179cdb48\") " pod="kuadrant-system/authorino-6d4dfc649f-wq8h4"
Sep 14 01:42:16.272463 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.272396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48a8315b-780b-448f-8ce5-e57d179cdb48-tls-cert\") pod \"authorino-6d4dfc649f-wq8h4\" (UID: \"48a8315b-780b-448f-8ce5-e57d179cdb48\") " pod="kuadrant-system/authorino-6d4dfc649f-wq8h4"
Sep 14 01:42:16.373958 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.373849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48a8315b-780b-448f-8ce5-e57d179cdb48-tls-cert\") pod \"authorino-6d4dfc649f-wq8h4\" (UID: \"48a8315b-780b-448f-8ce5-e57d179cdb48\") " pod="kuadrant-system/authorino-6d4dfc649f-wq8h4"
Sep 14 01:42:16.374118 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.373962    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxgwl\" (UniqueName: \"kubernetes.io/projected/48a8315b-780b-448f-8ce5-e57d179cdb48-kube-api-access-wxgwl\") pod \"authorino-6d4dfc649f-wq8h4\" (UID: \"48a8315b-780b-448f-8ce5-e57d179cdb48\") " pod="kuadrant-system/authorino-6d4dfc649f-wq8h4"
Sep 14 01:42:16.376528 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.376505    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48a8315b-780b-448f-8ce5-e57d179cdb48-tls-cert\") pod \"authorino-6d4dfc649f-wq8h4\" (UID: \"48a8315b-780b-448f-8ce5-e57d179cdb48\") " pod="kuadrant-system/authorino-6d4dfc649f-wq8h4"
Sep 14 01:42:16.380557 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.380528    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxgwl\" (UniqueName: \"kubernetes.io/projected/48a8315b-780b-448f-8ce5-e57d179cdb48-kube-api-access-wxgwl\") pod \"authorino-6d4dfc649f-wq8h4\" (UID: \"48a8315b-780b-448f-8ce5-e57d179cdb48\") " pod="kuadrant-system/authorino-6d4dfc649f-wq8h4"
Sep 14 01:42:16.985046 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.985019    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-v5pt4"
Sep 14 01:42:16.985266 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.985019    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-wq8h4"
Sep 14 01:42:16.992311 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.992287    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-v5pt4"
Sep 14 01:42:16.995407 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:16.995387    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-wq8h4"
Sep 14 01:42:17.079150 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:17.079120    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfmlg\" (UniqueName: \"kubernetes.io/projected/616f7462-efb8-43e6-858c-d294cf2e59bd-kube-api-access-sfmlg\") pod \"616f7462-efb8-43e6-858c-d294cf2e59bd\" (UID: \"616f7462-efb8-43e6-858c-d294cf2e59bd\") "
Sep 14 01:42:17.079583 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:17.079203    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48a8315b-780b-448f-8ce5-e57d179cdb48-tls-cert\") pod \"48a8315b-780b-448f-8ce5-e57d179cdb48\" (UID: \"48a8315b-780b-448f-8ce5-e57d179cdb48\") "
Sep 14 01:42:17.079583 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:17.079248    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxgwl\" (UniqueName: \"kubernetes.io/projected/48a8315b-780b-448f-8ce5-e57d179cdb48-kube-api-access-wxgwl\") pod \"48a8315b-780b-448f-8ce5-e57d179cdb48\" (UID: \"48a8315b-780b-448f-8ce5-e57d179cdb48\") "
Sep 14 01:42:17.081414 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:17.081388    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48a8315b-780b-448f-8ce5-e57d179cdb48-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "48a8315b-780b-448f-8ce5-e57d179cdb48" (UID: "48a8315b-780b-448f-8ce5-e57d179cdb48"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:42:17.081540 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:17.081412    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/616f7462-efb8-43e6-858c-d294cf2e59bd-kube-api-access-sfmlg" (OuterVolumeSpecName: "kube-api-access-sfmlg") pod "616f7462-efb8-43e6-858c-d294cf2e59bd" (UID: "616f7462-efb8-43e6-858c-d294cf2e59bd"). InnerVolumeSpecName "kube-api-access-sfmlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:42:17.081540 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:17.081439    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48a8315b-780b-448f-8ce5-e57d179cdb48-kube-api-access-wxgwl" (OuterVolumeSpecName: "kube-api-access-wxgwl") pod "48a8315b-780b-448f-8ce5-e57d179cdb48" (UID: "48a8315b-780b-448f-8ce5-e57d179cdb48"). InnerVolumeSpecName "kube-api-access-wxgwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:42:17.180774 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:17.180736    2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48a8315b-780b-448f-8ce5-e57d179cdb48-tls-cert\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:42:17.180774 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:17.180765    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxgwl\" (UniqueName: \"kubernetes.io/projected/48a8315b-780b-448f-8ce5-e57d179cdb48-kube-api-access-wxgwl\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:42:17.180774 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:17.180777    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfmlg\" (UniqueName: \"kubernetes.io/projected/616f7462-efb8-43e6-858c-d294cf2e59bd-kube-api-access-sfmlg\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:42:17.990654 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:17.990626    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-v5pt4"
Sep 14 01:42:17.990833 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:17.990626    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-wq8h4"
Sep 14 01:42:18.024147 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:18.024086    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-wq8h4"]
Sep 14 01:42:18.026156 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:18.026128    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-wq8h4"]
Sep 14 01:42:18.044070 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:18.044032    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-v5pt4"]
Sep 14 01:42:18.047609 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:18.047582    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-v5pt4"]
Sep 14 01:42:19.053142 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:19.053113    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48a8315b-780b-448f-8ce5-e57d179cdb48" path="/var/lib/kubelet/pods/48a8315b-780b-448f-8ce5-e57d179cdb48/volumes"
Sep 14 01:42:19.053508 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:42:19.053339    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="616f7462-efb8-43e6-858c-d294cf2e59bd" path="/var/lib/kubelet/pods/616f7462-efb8-43e6-858c-d294cf2e59bd/volumes"
Sep 14 01:43:00.166983 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:00.166946    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:43:17.375791 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:17.375758    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:43:25.957885 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:25.957847    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:43:45.961861 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:45.961821    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:43:56.896360 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:56.896326    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"]
Sep 14 01:43:56.899732 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:56.899711    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:56.902032 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:56.902006    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 14 01:43:56.902032 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:56.902020    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 14 01:43:56.902875 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:56.902858    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tpvrt\""
Sep 14 01:43:56.902977 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:56.902900    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 14 01:43:56.909679 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:56.909655    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"]
Sep 14 01:43:56.962598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:56.962558    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:56.962598 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:56.962598    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:56.962821 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:56.962627    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:56.962821 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:56.962697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:56.962821 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:56.962752    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z66cb\" (UniqueName: \"kubernetes.io/projected/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-kube-api-access-z66cb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:56.962961 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:56.962819    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:57.064114 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:57.064078    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:57.064114 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:57.064120    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:57.064332 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:57.064157    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z66cb\" (UniqueName: \"kubernetes.io/projected/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-kube-api-access-z66cb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:57.064332 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:57.064187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:57.064332 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:57.064214    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:57.064332 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:57.064240    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:57.064563 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:57.064541    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:57.064600 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:57.064578    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:57.064600 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:57.064594    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:57.066577 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:57.066553    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:57.066856 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:57.066839    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:57.071681 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:57.071653    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z66cb\" (UniqueName: \"kubernetes.io/projected/6f4e7b9a-52f2-414b-a479-7ac5e92977dd-kube-api-access-z66cb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k2x95\" (UID: \"6f4e7b9a-52f2-414b-a479-7ac5e92977dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:57.211741 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:57.211704    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:43:57.482511 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:57.482484    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"]
Sep 14 01:43:57.484327 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:43:57.484298    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f4e7b9a_52f2_414b_a479_7ac5e92977dd.slice/crio-143d111a01a2d90c2d5c694b25c1b9f5a2f0503108fc4632e18c6a843c3d5c5e WatchSource:0}: Error finding container 143d111a01a2d90c2d5c694b25c1b9f5a2f0503108fc4632e18c6a843c3d5c5e: Status 404 returned error can't find the container with id 143d111a01a2d90c2d5c694b25c1b9f5a2f0503108fc4632e18c6a843c3d5c5e
Sep 14 01:43:57.525405 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:57.525370    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95" event={"ID":"6f4e7b9a-52f2-414b-a479-7ac5e92977dd","Type":"ContainerStarted","Data":"143d111a01a2d90c2d5c694b25c1b9f5a2f0503108fc4632e18c6a843c3d5c5e"}
Sep 14 01:43:58.161469 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:58.161429    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:43:59.541441 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:59.541402    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95" event={"ID":"6f4e7b9a-52f2-414b-a479-7ac5e92977dd","Type":"ContainerStarted","Data":"2a6bf67147e1d1526141663c3517984b133d618c7e0ffc2a22ace64b5902768c"}
Sep 14 01:43:59.557493 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:43:59.557439    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95" podStartSLOduration=2.574543529 podStartE2EDuration="3.557423216s" totalImagesPullingTime="982.879687ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:43:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:43:59.555786895 +0000 UTC m=+799.036237718" watchObservedRunningTime="2026-09-14 01:43:59.557423216 +0000 UTC m=+799.037874075"
Sep 14 01:44:07.212713 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:07.212622    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:44:07.212713 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:07.212698    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:44:07.225435 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:07.225407    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:44:07.593584 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:07.593501    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k2x95"
Sep 14 01:44:14.753571 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:14.753525    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:44:18.007774 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.007742    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"]
Sep 14 01:44:18.011490 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.011464    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.013604 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.013580    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 14 01:44:18.020626 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.020595    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"]
Sep 14 01:44:18.145126 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.145084    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.145126 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.145133    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.145388 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.145179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.145388 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.145203    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.145388 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.145229    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nnbj\" (UniqueName: \"kubernetes.io/projected/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-kube-api-access-4nnbj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.145388 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.145340    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.246012 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.245975    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.246012 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.246016    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.246243 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.246068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.246243 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.246085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.246243 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.246115    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nnbj\" (UniqueName: \"kubernetes.io/projected/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-kube-api-access-4nnbj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.246243 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.246196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.246455 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.246431    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.246511 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.246458    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.246511 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.246491    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.248424 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.248397    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.248710 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.248689    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.253479 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.253453    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nnbj\" (UniqueName: \"kubernetes.io/projected/7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd-kube-api-access-4nnbj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clwbc\" (UID: \"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.323414 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.323317    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:18.598900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.598871    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"]
Sep 14 01:44:18.601022 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:44:18.600992    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e6b3d50_75f6_4ddf_a8c8_c4ad3a1769fd.slice/crio-a5ff7f1136b1819b26fba08dec85988238b6d9d642665f1f5572f4fed32c4c9d WatchSource:0}: Error finding container a5ff7f1136b1819b26fba08dec85988238b6d9d642665f1f5572f4fed32c4c9d: Status 404 returned error can't find the container with id a5ff7f1136b1819b26fba08dec85988238b6d9d642665f1f5572f4fed32c4c9d
Sep 14 01:44:18.642504 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:18.642466    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc" event={"ID":"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd","Type":"ContainerStarted","Data":"a5ff7f1136b1819b26fba08dec85988238b6d9d642665f1f5572f4fed32c4c9d"}
Sep 14 01:44:19.649651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:19.649611    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc" event={"ID":"7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd","Type":"ContainerStarted","Data":"b022262849af4351198047e2e8d1885a1a43e6f7d5e257c98bcd59f1a02ede00"}
Sep 14 01:44:19.666808 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:19.666758    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc" podStartSLOduration=2.485977577 podStartE2EDuration="2.666740025s" totalImagesPullingTime="180.762448ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:44:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:44:19.66602352 +0000 UTC m=+819.146474342" watchObservedRunningTime="2026-09-14 01:44:19.666740025 +0000 UTC m=+819.147190828"
Sep 14 01:44:19.962709 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:19.962665    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:44:28.323458 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:28.323416    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:28.324046 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:28.323494    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:28.336257 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:28.336226    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:28.707716 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:28.707685    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clwbc"
Sep 14 01:44:31.595663 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.595627    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"]
Sep 14 01:44:31.599791 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.599772    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.601920 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.601891    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 14 01:44:31.608953 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.608922    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"]
Sep 14 01:44:31.766475 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.766429    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c72667e5-4930-432d-b8fc-7e5ae7202b0c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.766656 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.766498    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c72667e5-4930-432d-b8fc-7e5ae7202b0c-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.766656 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.766515    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c72667e5-4930-432d-b8fc-7e5ae7202b0c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.766656 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.766557    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sdfp\" (UniqueName: \"kubernetes.io/projected/c72667e5-4930-432d-b8fc-7e5ae7202b0c-kube-api-access-2sdfp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.766656 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.766624    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c72667e5-4930-432d-b8fc-7e5ae7202b0c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.766800 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.766679    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c72667e5-4930-432d-b8fc-7e5ae7202b0c-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.867683 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.867586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sdfp\" (UniqueName: \"kubernetes.io/projected/c72667e5-4930-432d-b8fc-7e5ae7202b0c-kube-api-access-2sdfp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.867683 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.867636    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c72667e5-4930-432d-b8fc-7e5ae7202b0c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.867683 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.867659    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c72667e5-4930-432d-b8fc-7e5ae7202b0c-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.867683 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.867687    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c72667e5-4930-432d-b8fc-7e5ae7202b0c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.868059 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.867733    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c72667e5-4930-432d-b8fc-7e5ae7202b0c-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.868059 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.867747    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c72667e5-4930-432d-b8fc-7e5ae7202b0c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.868176 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.868102    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c72667e5-4930-432d-b8fc-7e5ae7202b0c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.868176 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.868160    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c72667e5-4930-432d-b8fc-7e5ae7202b0c-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.868258 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.868195    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c72667e5-4930-432d-b8fc-7e5ae7202b0c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.870119 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.870089    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c72667e5-4930-432d-b8fc-7e5ae7202b0c-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.870279 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.870259    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c72667e5-4930-432d-b8fc-7e5ae7202b0c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.875061 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.875036    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sdfp\" (UniqueName: \"kubernetes.io/projected/c72667e5-4930-432d-b8fc-7e5ae7202b0c-kube-api-access-2sdfp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2\" (UID: \"c72667e5-4930-432d-b8fc-7e5ae7202b0c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:31.912020 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:31.911976    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:32.184660 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:32.184632    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"]
Sep 14 01:44:32.187092 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:44:32.187065    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc72667e5_4930_432d_b8fc_7e5ae7202b0c.slice/crio-9513b2f4fd8f15ed46924ebfe3167daeba559e7992f51cb3f02f617ae3b61a79 WatchSource:0}: Error finding container 9513b2f4fd8f15ed46924ebfe3167daeba559e7992f51cb3f02f617ae3b61a79: Status 404 returned error can't find the container with id 9513b2f4fd8f15ed46924ebfe3167daeba559e7992f51cb3f02f617ae3b61a79
Sep 14 01:44:32.720555 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:32.720524    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2" event={"ID":"c72667e5-4930-432d-b8fc-7e5ae7202b0c","Type":"ContainerStarted","Data":"7463a96fc10498de26918c3a1f3fa7d0ce23f06b271d902d86c301a60ca4e305"}
Sep 14 01:44:32.721028 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:32.720563    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2" event={"ID":"c72667e5-4930-432d-b8fc-7e5ae7202b0c","Type":"ContainerStarted","Data":"9513b2f4fd8f15ed46924ebfe3167daeba559e7992f51cb3f02f617ae3b61a79"}
Sep 14 01:44:32.735590 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:32.735540    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2" podStartSLOduration=1.542313891 podStartE2EDuration="1.735520991s" totalImagesPullingTime="193.2071ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:44:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:44:32.734513711 +0000 UTC m=+832.214964539" watchObservedRunningTime="2026-09-14 01:44:32.735520991 +0000 UTC m=+832.215971811"
Sep 14 01:44:34.061201 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:34.061160    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:44:41.912718 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:41.912657    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:41.912718 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:41.912722    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:41.925479 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:41.925448    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:44:42.803608 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:44:42.803577    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2"
Sep 14 01:45:41.061951 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:41.061841    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 01:45:41.061951 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:41.061932    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 01:45:43.059459 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:43.059427    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5c44878777-wbhzm"]
Sep 14 01:45:43.062679 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:43.062662    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c44878777-wbhzm"
Sep 14 01:45:43.065104 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:43.065076    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 14 01:45:43.065104 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:43.065078    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gr8lh\""
Sep 14 01:45:43.072018 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:43.071995    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c44878777-wbhzm"]
Sep 14 01:45:43.124626 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:43.124594    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/71c7e401-c498-4d13-b69b-6a4d718ee4a6-tls-cert\") pod \"authorino-5c44878777-wbhzm\" (UID: \"71c7e401-c498-4d13-b69b-6a4d718ee4a6\") " pod="kuadrant-system/authorino-5c44878777-wbhzm"
Sep 14 01:45:43.124822 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:43.124726    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnjkm\" (UniqueName: \"kubernetes.io/projected/71c7e401-c498-4d13-b69b-6a4d718ee4a6-kube-api-access-vnjkm\") pod \"authorino-5c44878777-wbhzm\" (UID: \"71c7e401-c498-4d13-b69b-6a4d718ee4a6\") " pod="kuadrant-system/authorino-5c44878777-wbhzm"
Sep 14 01:45:43.225538 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:43.225494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjkm\" (UniqueName: \"kubernetes.io/projected/71c7e401-c498-4d13-b69b-6a4d718ee4a6-kube-api-access-vnjkm\") pod \"authorino-5c44878777-wbhzm\" (UID: \"71c7e401-c498-4d13-b69b-6a4d718ee4a6\") " pod="kuadrant-system/authorino-5c44878777-wbhzm"
Sep 14 01:45:43.225722 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:43.225596    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/71c7e401-c498-4d13-b69b-6a4d718ee4a6-tls-cert\") pod \"authorino-5c44878777-wbhzm\" (UID: \"71c7e401-c498-4d13-b69b-6a4d718ee4a6\") " pod="kuadrant-system/authorino-5c44878777-wbhzm"
Sep 14 01:45:43.228132 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:43.228113    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/71c7e401-c498-4d13-b69b-6a4d718ee4a6-tls-cert\") pod \"authorino-5c44878777-wbhzm\" (UID: \"71c7e401-c498-4d13-b69b-6a4d718ee4a6\") " pod="kuadrant-system/authorino-5c44878777-wbhzm"
Sep 14 01:45:43.232482 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:43.232462    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnjkm\" (UniqueName: \"kubernetes.io/projected/71c7e401-c498-4d13-b69b-6a4d718ee4a6-kube-api-access-vnjkm\") pod \"authorino-5c44878777-wbhzm\" (UID: \"71c7e401-c498-4d13-b69b-6a4d718ee4a6\") " pod="kuadrant-system/authorino-5c44878777-wbhzm"
Sep 14 01:45:43.372508 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:43.372415    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c44878777-wbhzm"
Sep 14 01:45:43.648947 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:43.645392    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c44878777-wbhzm"]
Sep 14 01:45:44.117438 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:44.117397    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c44878777-wbhzm" event={"ID":"71c7e401-c498-4d13-b69b-6a4d718ee4a6","Type":"ContainerStarted","Data":"fc7514d3a09907a44aed5ddd459944b55229c1fd0581a05dece27ec4a98fe08f"}
Sep 14 01:45:45.125060 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:45.125021    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c44878777-wbhzm" event={"ID":"71c7e401-c498-4d13-b69b-6a4d718ee4a6","Type":"ContainerStarted","Data":"69c1f57fe92d99698a3ec78face77f923223a04155a5687d56704b96a64da208"}
Sep 14 01:45:45.142396 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:45:45.142322    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5c44878777-wbhzm" podStartSLOduration=1.305330425 podStartE2EDuration="2.142306628s" totalImagesPullingTime="836.976203ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:45:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:45:45.140056309 +0000 UTC m=+904.620507131" watchObservedRunningTime="2026-09-14 01:45:45.142306628 +0000 UTC m=+904.622757443"
Sep 14 01:46:06.945637 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:06.945602    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7455-c43afe"]
Sep 14 01:46:06.949390 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:06.949370    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-c43afe"
Sep 14 01:46:06.951347 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:06.951322    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-74qx8\""
Sep 14 01:46:06.955708 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:06.955678    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7455-c43afe"]
Sep 14 01:46:07.047132 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:07.047086    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc5bb\" (UniqueName: \"kubernetes.io/projected/31ed6a13-9d04-47cd-b67b-5f49d1e72e32-kube-api-access-kc5bb\") pod \"test-curl-7455-c43afe\" (UID: \"31ed6a13-9d04-47cd-b67b-5f49d1e72e32\") " pod="opendatahub/test-curl-7455-c43afe"
Sep 14 01:46:07.148076 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:07.148036    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc5bb\" (UniqueName: \"kubernetes.io/projected/31ed6a13-9d04-47cd-b67b-5f49d1e72e32-kube-api-access-kc5bb\") pod \"test-curl-7455-c43afe\" (UID: \"31ed6a13-9d04-47cd-b67b-5f49d1e72e32\") " pod="opendatahub/test-curl-7455-c43afe"
Sep 14 01:46:07.155467 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:07.155438    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc5bb\" (UniqueName: \"kubernetes.io/projected/31ed6a13-9d04-47cd-b67b-5f49d1e72e32-kube-api-access-kc5bb\") pod \"test-curl-7455-c43afe\" (UID: \"31ed6a13-9d04-47cd-b67b-5f49d1e72e32\") " pod="opendatahub/test-curl-7455-c43afe"
Sep 14 01:46:07.260624 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:07.260594    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-c43afe"
Sep 14 01:46:07.535325 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:07.535297    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7455-c43afe"]
Sep 14 01:46:07.537036 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:46:07.537008    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31ed6a13_9d04_47cd_b67b_5f49d1e72e32.slice/crio-509fa0ba3e714215dfbb523ba2f8a3e9fbc4ea1f10ff8fe902613fa6a45e936e WatchSource:0}: Error finding container 509fa0ba3e714215dfbb523ba2f8a3e9fbc4ea1f10ff8fe902613fa6a45e936e: Status 404 returned error can't find the container with id 509fa0ba3e714215dfbb523ba2f8a3e9fbc4ea1f10ff8fe902613fa6a45e936e
Sep 14 01:46:07.539041 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:07.539021    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 01:46:08.255327 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:08.255285    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7455-c43afe" event={"ID":"31ed6a13-9d04-47cd-b67b-5f49d1e72e32","Type":"ContainerStarted","Data":"509fa0ba3e714215dfbb523ba2f8a3e9fbc4ea1f10ff8fe902613fa6a45e936e"}
Sep 14 01:46:08.576228 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:08.576148    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:46:11.273348 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:11.273307    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7455-c43afe" event={"ID":"31ed6a13-9d04-47cd-b67b-5f49d1e72e32","Type":"ContainerStarted","Data":"17f7360e7c8e1a0bf1628e130432b10f2248f239e2ce881151126a43550fe902"}
Sep 14 01:46:11.289021 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:11.288969    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7455-c43afe" podStartSLOduration=2.138197388 podStartE2EDuration="5.288952901s" totalImagesPullingTime="3.150755513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:46:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:46:11.288091437 +0000 UTC m=+930.768542261" watchObservedRunningTime="2026-09-14 01:46:11.288952901 +0000 UTC m=+930.769403721"
Sep 14 01:46:11.926746 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:11.926705    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7455-c43afe"]
Sep 14 01:46:11.930907 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:11.930877    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7455-c43afe"]
Sep 14 01:46:12.051507 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:12.051422    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7455-bc6ea7"]
Sep 14 01:46:12.051819 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:12.051806    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31ed6a13-9d04-47cd-b67b-5f49d1e72e32" containerName="test-curl-7455-c43afe"
Sep 14 01:46:12.051863 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:12.051820    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="31ed6a13-9d04-47cd-b67b-5f49d1e72e32" containerName="test-curl-7455-c43afe"
Sep 14 01:46:12.051938 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:12.051927    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="31ed6a13-9d04-47cd-b67b-5f49d1e72e32" containerName="test-curl-7455-c43afe"
Sep 14 01:46:12.055214 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:12.055199    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-bc6ea7"
Sep 14 01:46:12.060687 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:12.060660    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7455-bc6ea7"]
Sep 14 01:46:12.094023 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:12.093990    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ln8b\" (UniqueName: \"kubernetes.io/projected/aa0e91b4-16d0-48bf-821d-576cfe4cc02f-kube-api-access-2ln8b\") pod \"test-curl-7455-bc6ea7\" (UID: \"aa0e91b4-16d0-48bf-821d-576cfe4cc02f\") " pod="opendatahub/test-curl-7455-bc6ea7"
Sep 14 01:46:12.195160 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:12.195108    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ln8b\" (UniqueName: \"kubernetes.io/projected/aa0e91b4-16d0-48bf-821d-576cfe4cc02f-kube-api-access-2ln8b\") pod \"test-curl-7455-bc6ea7\" (UID: \"aa0e91b4-16d0-48bf-821d-576cfe4cc02f\") " pod="opendatahub/test-curl-7455-bc6ea7"
Sep 14 01:46:12.202127 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:12.202099    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ln8b\" (UniqueName: \"kubernetes.io/projected/aa0e91b4-16d0-48bf-821d-576cfe4cc02f-kube-api-access-2ln8b\") pod \"test-curl-7455-bc6ea7\" (UID: \"aa0e91b4-16d0-48bf-821d-576cfe4cc02f\") " pod="opendatahub/test-curl-7455-bc6ea7"
Sep 14 01:46:12.278710 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:46:12.278680    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7455-c43afe" container="test-curl-7455-c43afe"
Sep 14 01:46:12.366513 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:12.366425    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-bc6ea7"
Sep 14 01:46:12.635777 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:12.635751    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7455-bc6ea7"]
Sep 14 01:46:12.637602 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:46:12.637575    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa0e91b4_16d0_48bf_821d_576cfe4cc02f.slice/crio-34130da2d7e02297b28eef072f4b04549b1437ba1f155110b5a7a4f0fa9a8e51 WatchSource:0}: Error finding container 34130da2d7e02297b28eef072f4b04549b1437ba1f155110b5a7a4f0fa9a8e51: Status 404 returned error can't find the container with id 34130da2d7e02297b28eef072f4b04549b1437ba1f155110b5a7a4f0fa9a8e51
Sep 14 01:46:13.286239 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:13.286203    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7455-bc6ea7" event={"ID":"aa0e91b4-16d0-48bf-821d-576cfe4cc02f","Type":"ContainerStarted","Data":"69776f7be194df49cbfa3ad957358aa8f558ecf99ed62f814edd4d465546c79b"}
Sep 14 01:46:13.286239 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:13.286245    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7455-bc6ea7" event={"ID":"aa0e91b4-16d0-48bf-821d-576cfe4cc02f","Type":"ContainerStarted","Data":"34130da2d7e02297b28eef072f4b04549b1437ba1f155110b5a7a4f0fa9a8e51"}
Sep 14 01:46:13.286772 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:13.286290    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7455-c43afe" podUID="31ed6a13-9d04-47cd-b67b-5f49d1e72e32" containerName="test-curl-7455-c43afe" containerID="cri-o://17f7360e7c8e1a0bf1628e130432b10f2248f239e2ce881151126a43550fe902" gracePeriod=2
Sep 14 01:46:13.299330 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:13.299281    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7455-bc6ea7" podStartSLOduration=1.299255267 podStartE2EDuration="1.299255267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:46:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:46:13.298860853 +0000 UTC m=+932.779311674" watchObservedRunningTime="2026-09-14 01:46:13.299255267 +0000 UTC m=+932.779706122"
Sep 14 01:46:13.915202 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:13.915168    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7455-bc6ea7"]
Sep 14 01:46:13.917616 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:13.917587    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7455-bc6ea7"]
Sep 14 01:46:14.292103 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:46:14.292064    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7455-bc6ea7" container="test-curl-7455-bc6ea7"
Sep 14 01:46:15.297440 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:15.297409    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7455-bc6ea7" podUID="aa0e91b4-16d0-48bf-821d-576cfe4cc02f" containerName="test-curl-7455-bc6ea7" containerID="cri-o://69776f7be194df49cbfa3ad957358aa8f558ecf99ed62f814edd4d465546c79b" gracePeriod=2
Sep 14 01:46:15.467580 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:15.467549    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-c43afe"
Sep 14 01:46:15.523830 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:15.523794    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kc5bb\" (UniqueName: \"kubernetes.io/projected/31ed6a13-9d04-47cd-b67b-5f49d1e72e32-kube-api-access-kc5bb\") pod \"31ed6a13-9d04-47cd-b67b-5f49d1e72e32\" (UID: \"31ed6a13-9d04-47cd-b67b-5f49d1e72e32\") "
Sep 14 01:46:15.526087 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:15.526062    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31ed6a13-9d04-47cd-b67b-5f49d1e72e32-kube-api-access-kc5bb" (OuterVolumeSpecName: "kube-api-access-kc5bb") pod "31ed6a13-9d04-47cd-b67b-5f49d1e72e32" (UID: "31ed6a13-9d04-47cd-b67b-5f49d1e72e32"). InnerVolumeSpecName "kube-api-access-kc5bb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:46:15.624720 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:15.624624    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kc5bb\" (UniqueName: \"kubernetes.io/projected/31ed6a13-9d04-47cd-b67b-5f49d1e72e32-kube-api-access-kc5bb\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:46:16.304729 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:16.304688    2581 generic.go:356] "Generic (PLEG): container finished" podID="31ed6a13-9d04-47cd-b67b-5f49d1e72e32" containerID="17f7360e7c8e1a0bf1628e130432b10f2248f239e2ce881151126a43550fe902" exitCode=137
Sep 14 01:46:16.305191 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:16.304745    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-c43afe"
Sep 14 01:46:16.305191 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:16.304747    2581 scope.go:117] "RemoveContainer" containerID="17f7360e7c8e1a0bf1628e130432b10f2248f239e2ce881151126a43550fe902"
Sep 14 01:46:16.314590 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:46:16.314551    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7455-c43afe\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="31ed6a13-9d04-47cd-b67b-5f49d1e72e32" pod="opendatahub/test-curl-7455-c43afe"
Sep 14 01:46:16.315028 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:16.314993    2581 scope.go:117] "RemoveContainer" containerID="17f7360e7c8e1a0bf1628e130432b10f2248f239e2ce881151126a43550fe902"
Sep 14 01:46:16.315417 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:46:16.315386    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17f7360e7c8e1a0bf1628e130432b10f2248f239e2ce881151126a43550fe902\": container with ID starting with 17f7360e7c8e1a0bf1628e130432b10f2248f239e2ce881151126a43550fe902 not found: ID does not exist" containerID="17f7360e7c8e1a0bf1628e130432b10f2248f239e2ce881151126a43550fe902"
Sep 14 01:46:16.315514 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:16.315429    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17f7360e7c8e1a0bf1628e130432b10f2248f239e2ce881151126a43550fe902"} err="failed to get container status \"17f7360e7c8e1a0bf1628e130432b10f2248f239e2ce881151126a43550fe902\": rpc error: code = NotFound desc = could not find container \"17f7360e7c8e1a0bf1628e130432b10f2248f239e2ce881151126a43550fe902\": container with ID starting with 17f7360e7c8e1a0bf1628e130432b10f2248f239e2ce881151126a43550fe902 not found: ID does not exist"
Sep 14 01:46:17.056059 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:17.056023    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31ed6a13-9d04-47cd-b67b-5f49d1e72e32" path="/var/lib/kubelet/pods/31ed6a13-9d04-47cd-b67b-5f49d1e72e32/volumes"
Sep 14 01:46:17.497878 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:17.497843    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-bc6ea7"
Sep 14 01:46:17.542901 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:17.542869    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ln8b\" (UniqueName: \"kubernetes.io/projected/aa0e91b4-16d0-48bf-821d-576cfe4cc02f-kube-api-access-2ln8b\") pod \"aa0e91b4-16d0-48bf-821d-576cfe4cc02f\" (UID: \"aa0e91b4-16d0-48bf-821d-576cfe4cc02f\") "
Sep 14 01:46:17.545165 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:17.545134    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa0e91b4-16d0-48bf-821d-576cfe4cc02f-kube-api-access-2ln8b" (OuterVolumeSpecName: "kube-api-access-2ln8b") pod "aa0e91b4-16d0-48bf-821d-576cfe4cc02f" (UID: "aa0e91b4-16d0-48bf-821d-576cfe4cc02f"). InnerVolumeSpecName "kube-api-access-2ln8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:46:17.644028 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:17.643907    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ln8b\" (UniqueName: \"kubernetes.io/projected/aa0e91b4-16d0-48bf-821d-576cfe4cc02f-kube-api-access-2ln8b\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:46:18.320638 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:18.320599    2581 generic.go:356] "Generic (PLEG): container finished" podID="aa0e91b4-16d0-48bf-821d-576cfe4cc02f" containerID="69776f7be194df49cbfa3ad957358aa8f558ecf99ed62f814edd4d465546c79b" exitCode=137
Sep 14 01:46:18.320817 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:18.320645    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7455-bc6ea7"
Sep 14 01:46:18.320817 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:18.320665    2581 scope.go:117] "RemoveContainer" containerID="69776f7be194df49cbfa3ad957358aa8f558ecf99ed62f814edd4d465546c79b"
Sep 14 01:46:18.331290 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:46:18.331256    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7455-bc6ea7\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="aa0e91b4-16d0-48bf-821d-576cfe4cc02f" pod="opendatahub/test-curl-7455-bc6ea7"
Sep 14 01:46:18.333543 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:18.333274    2581 scope.go:117] "RemoveContainer" containerID="69776f7be194df49cbfa3ad957358aa8f558ecf99ed62f814edd4d465546c79b"
Sep 14 01:46:18.333972 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:46:18.333948    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69776f7be194df49cbfa3ad957358aa8f558ecf99ed62f814edd4d465546c79b\": container with ID starting with 69776f7be194df49cbfa3ad957358aa8f558ecf99ed62f814edd4d465546c79b not found: ID does not exist" containerID="69776f7be194df49cbfa3ad957358aa8f558ecf99ed62f814edd4d465546c79b"
Sep 14 01:46:18.334068 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:18.333979    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69776f7be194df49cbfa3ad957358aa8f558ecf99ed62f814edd4d465546c79b"} err="failed to get container status \"69776f7be194df49cbfa3ad957358aa8f558ecf99ed62f814edd4d465546c79b\": rpc error: code = NotFound desc = could not find container \"69776f7be194df49cbfa3ad957358aa8f558ecf99ed62f814edd4d465546c79b\": container with ID starting with 69776f7be194df49cbfa3ad957358aa8f558ecf99ed62f814edd4d465546c79b not found: ID does not exist"
Sep 14 01:46:19.053821 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:19.053785    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa0e91b4-16d0-48bf-821d-576cfe4cc02f" path="/var/lib/kubelet/pods/aa0e91b4-16d0-48bf-821d-576cfe4cc02f/volumes"
Sep 14 01:46:20.110184 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.110151    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"]
Sep 14 01:46:20.110742 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.110717    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa0e91b4-16d0-48bf-821d-576cfe4cc02f" containerName="test-curl-7455-bc6ea7"
Sep 14 01:46:20.110827 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.110745    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa0e91b4-16d0-48bf-821d-576cfe4cc02f" containerName="test-curl-7455-bc6ea7"
Sep 14 01:46:20.110902 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.110889    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa0e91b4-16d0-48bf-821d-576cfe4cc02f" containerName="test-curl-7455-bc6ea7"
Sep 14 01:46:20.115543 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.115515    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:20.117702 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.117674    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e3d76dcd-serving-cert\""
Sep 14 01:46:20.117941 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.117857    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8hsv9\""
Sep 14 01:46:20.117941 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.117935    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 01:46:20.118476 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.118453    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 01:46:20.118692 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.118466    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 01:46:20.122779 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.122749    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"]
Sep 14 01:46:20.167234 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.167184    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbrmc\" (UniqueName: \"kubernetes.io/projected/0389bac9-b8c8-4d6b-95dc-7318bc84386d-kube-api-access-zbrmc\") pod \"maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:20.167234 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.167230    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0389bac9-b8c8-4d6b-95dc-7318bc84386d-maas-api-tls\") pod \"maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:20.167480 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.167246    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0389bac9-b8c8-4d6b-95dc-7318bc84386d-metrics-certs\") pod \"maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:20.167480 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.167377    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0389bac9-b8c8-4d6b-95dc-7318bc84386d-openshift-service-ca\") pod \"maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:20.268573 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.268534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0389bac9-b8c8-4d6b-95dc-7318bc84386d-openshift-service-ca\") pod \"maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:20.268786 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.268603    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbrmc\" (UniqueName: \"kubernetes.io/projected/0389bac9-b8c8-4d6b-95dc-7318bc84386d-kube-api-access-zbrmc\") pod \"maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:20.268786 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.268623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0389bac9-b8c8-4d6b-95dc-7318bc84386d-maas-api-tls\") pod \"maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:20.268786 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.268639    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0389bac9-b8c8-4d6b-95dc-7318bc84386d-metrics-certs\") pod \"maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:20.268786 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:46:20.268773    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-serving-cert: secret "maas-api-e2e-ait-e3d76dcd-serving-cert" not found
Sep 14 01:46:20.269055 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:46:20.268848    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0389bac9-b8c8-4d6b-95dc-7318bc84386d-maas-api-tls podName:0389bac9-b8c8-4d6b-95dc-7318bc84386d nodeName:}" failed. No retries permitted until 2026-09-14 01:46:20.768826916 +0000 UTC m=+940.249277731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0389bac9-b8c8-4d6b-95dc-7318bc84386d-maas-api-tls") pod "maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8" (UID: "0389bac9-b8c8-4d6b-95dc-7318bc84386d") : secret "maas-api-e2e-ait-e3d76dcd-serving-cert" not found
Sep 14 01:46:20.269358 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.269339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0389bac9-b8c8-4d6b-95dc-7318bc84386d-openshift-service-ca\") pod \"maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:20.271093 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.271072    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0389bac9-b8c8-4d6b-95dc-7318bc84386d-metrics-certs\") pod \"maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:20.275961 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.275940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbrmc\" (UniqueName: \"kubernetes.io/projected/0389bac9-b8c8-4d6b-95dc-7318bc84386d-kube-api-access-zbrmc\") pod \"maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:20.773828 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.773786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0389bac9-b8c8-4d6b-95dc-7318bc84386d-maas-api-tls\") pod \"maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:20.776270 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:20.776238    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0389bac9-b8c8-4d6b-95dc-7318bc84386d-maas-api-tls\") pod \"maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:21.022183 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.022147    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"]
Sep 14 01:46:21.026180 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.026077    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:21.028176 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.028152    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-bc2e37-serving-cert\""
Sep 14 01:46:21.030734 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.030707    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:21.043483 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.042384    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"]
Sep 14 01:46:21.076106 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.076061    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9909d893-a3d9-45ce-bd57-3a1ed91a344d-metrics-certs\") pod \"maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:21.076106 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.076111    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9909d893-a3d9-45ce-bd57-3a1ed91a344d-maas-api-tls\") pod \"maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:21.076335 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.076205    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcwgl\" (UniqueName: \"kubernetes.io/projected/9909d893-a3d9-45ce-bd57-3a1ed91a344d-kube-api-access-rcwgl\") pod \"maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:21.076335 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.076273    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9909d893-a3d9-45ce-bd57-3a1ed91a344d-openshift-service-ca\") pod \"maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:21.177108 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.177074    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9909d893-a3d9-45ce-bd57-3a1ed91a344d-openshift-service-ca\") pod \"maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:21.177608 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.177179    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9909d893-a3d9-45ce-bd57-3a1ed91a344d-metrics-certs\") pod \"maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:21.177608 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.177221    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9909d893-a3d9-45ce-bd57-3a1ed91a344d-maas-api-tls\") pod \"maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:21.177608 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.177273    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcwgl\" (UniqueName: \"kubernetes.io/projected/9909d893-a3d9-45ce-bd57-3a1ed91a344d-kube-api-access-rcwgl\") pod \"maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:21.178018 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.177998    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9909d893-a3d9-45ce-bd57-3a1ed91a344d-openshift-service-ca\") pod \"maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:21.179833 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.179808    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9909d893-a3d9-45ce-bd57-3a1ed91a344d-maas-api-tls\") pod \"maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:21.179970 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.179936    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9909d893-a3d9-45ce-bd57-3a1ed91a344d-metrics-certs\") pod \"maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:21.183712 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.183683    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcwgl\" (UniqueName: \"kubernetes.io/projected/9909d893-a3d9-45ce-bd57-3a1ed91a344d-kube-api-access-rcwgl\") pod \"maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:21.314947 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.314903    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"]
Sep 14 01:46:21.316645 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:46:21.316612    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0389bac9_b8c8_4d6b_95dc_7318bc84386d.slice/crio-1b519562a9af2ded48e369b27782397e96bfdd1d6d2516f5581d19ec53bd33fd WatchSource:0}: Error finding container 1b519562a9af2ded48e369b27782397e96bfdd1d6d2516f5581d19ec53bd33fd: Status 404 returned error can't find the container with id 1b519562a9af2ded48e369b27782397e96bfdd1d6d2516f5581d19ec53bd33fd
Sep 14 01:46:21.339994 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.339960    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8" event={"ID":"0389bac9-b8c8-4d6b-95dc-7318bc84386d","Type":"ContainerStarted","Data":"1b519562a9af2ded48e369b27782397e96bfdd1d6d2516f5581d19ec53bd33fd"}
Sep 14 01:46:21.342099 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.342076    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:21.615667 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:21.615640    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"]
Sep 14 01:46:21.617618 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:46:21.617588    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9909d893_a3d9_45ce_bd57_3a1ed91a344d.slice/crio-65e8af485c76104e7659399ffd8e3ff537871ff15e6055730c132cb5ac0ff151 WatchSource:0}: Error finding container 65e8af485c76104e7659399ffd8e3ff537871ff15e6055730c132cb5ac0ff151: Status 404 returned error can't find the container with id 65e8af485c76104e7659399ffd8e3ff537871ff15e6055730c132cb5ac0ff151
Sep 14 01:46:22.350966 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:22.350855    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7" event={"ID":"9909d893-a3d9-45ce-bd57-3a1ed91a344d","Type":"ContainerStarted","Data":"65e8af485c76104e7659399ffd8e3ff537871ff15e6055730c132cb5ac0ff151"}
Sep 14 01:46:24.365341 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:24.365292    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8" event={"ID":"0389bac9-b8c8-4d6b-95dc-7318bc84386d","Type":"ContainerStarted","Data":"84ed367b011060b129386030f4f3e37534a61acb8f23cfd1dcea19af30c4073a"}
Sep 14 01:46:24.365801 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:24.365402    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:24.367277 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:24.367253    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7" event={"ID":"9909d893-a3d9-45ce-bd57-3a1ed91a344d","Type":"ContainerStarted","Data":"cae769fd0b5c243f84fefad95a0a51c412ddc7adc472856f313593b7528e65b6"}
Sep 14 01:46:24.367411 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:24.367339    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:24.382738 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:24.382690    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8" podStartSLOduration=1.9620963329999999 podStartE2EDuration="4.382674059s" totalImagesPullingTime="2.420577726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:46:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:46:24.380298563 +0000 UTC m=+943.860749385" watchObservedRunningTime="2026-09-14 01:46:24.382674059 +0000 UTC m=+943.863124879"
Sep 14 01:46:24.396157 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:24.396106    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7" podStartSLOduration=1.270119238 podStartE2EDuration="3.396090394s" totalImagesPullingTime="2.125971156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:46:24.394534189 +0000 UTC m=+943.874985022" watchObservedRunningTime="2026-09-14 01:46:24.396090394 +0000 UTC m=+943.876541215"
Sep 14 01:46:25.468405 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:25.468351    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:46:30.378741 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:30.378707    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:46:30.379193 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:30.378795    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:39.749185 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.749151    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"]
Sep 14 01:46:39.755903 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.755878    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:39.757865 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.757817    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-321811-serving-cert\""
Sep 14 01:46:39.761535 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.761506    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"]
Sep 14 01:46:39.854757 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.854723    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-maas-api-tls\") pod \"maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:39.854974 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.854789    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-openshift-service-ca\") pod \"maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:39.854974 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.854820    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-metrics-certs\") pod \"maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:39.854974 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.854901    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dhq8\" (UniqueName: \"kubernetes.io/projected/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-kube-api-access-7dhq8\") pod \"maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:39.955670 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.955637    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dhq8\" (UniqueName: \"kubernetes.io/projected/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-kube-api-access-7dhq8\") pod \"maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:39.955860 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.955695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-maas-api-tls\") pod \"maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:39.955860 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.955743    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-openshift-service-ca\") pod \"maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:39.956054 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.955890    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-metrics-certs\") pod \"maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:39.956492 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.956468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-openshift-service-ca\") pod \"maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:39.958352 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.958321    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-maas-api-tls\") pod \"maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:39.958450 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.958378    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-metrics-certs\") pod \"maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:39.962803 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:39.962778    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dhq8\" (UniqueName: \"kubernetes.io/projected/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-kube-api-access-7dhq8\") pod \"maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:40.068050 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:40.067951    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:40.347522 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:40.347493    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"]
Sep 14 01:46:40.349668 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:46:40.349639    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2fc2b86_9eb8_4d3c_b27f_3c3fbb90d562.slice/crio-8ea9da3c3dbac4b071febd0db0c24a24c880cba2ac0962cd3bb9283a7ce97b1c WatchSource:0}: Error finding container 8ea9da3c3dbac4b071febd0db0c24a24c880cba2ac0962cd3bb9283a7ce97b1c: Status 404 returned error can't find the container with id 8ea9da3c3dbac4b071febd0db0c24a24c880cba2ac0962cd3bb9283a7ce97b1c
Sep 14 01:46:40.463842 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:40.463803    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c" event={"ID":"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562","Type":"ContainerStarted","Data":"8ea9da3c3dbac4b071febd0db0c24a24c880cba2ac0962cd3bb9283a7ce97b1c"}
Sep 14 01:46:41.357319 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:41.357228    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:46:41.470400 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:41.470354    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c" event={"ID":"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562","Type":"ContainerStarted","Data":"f7970f61dc4c3eee8694de5d2fbcb4afb599cb87900d656178401f7e6798e2e0"}
Sep 14 01:46:41.470571 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:41.470449    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:41.487139 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:41.487092    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c" podStartSLOduration=2.139719967 podStartE2EDuration="2.487074006s" totalImagesPullingTime="347.354039ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:46:41.484860755 +0000 UTC m=+960.965311573" watchObservedRunningTime="2026-09-14 01:46:41.487074006 +0000 UTC m=+960.967524826"
Sep 14 01:46:45.993121 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:45.993083    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"]
Sep 14 01:46:45.998518 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:45.998496    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.000479 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.000450    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-99c616\"/\"openshift-service-ca.crt\""
Sep 14 01:46:46.000479 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.000468    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-99c616\"/\"kube-root-ca.crt\""
Sep 14 01:46:46.001088 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.001060    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-99c616\"/\"default-dockercfg-pst88\""
Sep 14 01:46:46.001227 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.001142    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-99c616\"/\"simulator-w0-99c616-kserve-self-signed-certs\""
Sep 14 01:46:46.004679 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.004656    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"]
Sep 14 01:46:46.111800 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.111757    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-model-cache\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.111800 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.111799    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-tmp-dir\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.112113 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.111831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-home\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.112113 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.111887    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-dshm\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.112113 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.111966    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgt9b\" (UniqueName: \"kubernetes.io/projected/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-kube-api-access-sgt9b\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.112113 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.111994    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-tls-certs\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.213330 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.213290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-dshm\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.213514 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.213345    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgt9b\" (UniqueName: \"kubernetes.io/projected/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-kube-api-access-sgt9b\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.213514 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.213380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-tls-certs\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.213639 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.213597    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-model-cache\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.213639 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.213633    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-tmp-dir\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.213738 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.213656    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-home\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.214011 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.213988    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-model-cache\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.214157 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.214048    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-tmp-dir\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.214157 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.214076    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-home\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.215775 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.215754    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-dshm\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.215935 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.215900    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-tls-certs\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.220640 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.220617    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgt9b\" (UniqueName: \"kubernetes.io/projected/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-kube-api-access-sgt9b\") pod \"simulator-w0-99c616-kserve-868ff5d65-mrrrl\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") " pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.308884 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.308793    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:46.579642 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:46.579612    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"]
Sep 14 01:46:46.581282 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:46:46.581253    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8db41348_9d3a_46e4_bc62_58bb3ffa4bdc.slice/crio-ea03cb6088ffd193f18670385ca39844657f97a621543af7798c3dfd17b534be WatchSource:0}: Error finding container ea03cb6088ffd193f18670385ca39844657f97a621543af7798c3dfd17b534be: Status 404 returned error can't find the container with id ea03cb6088ffd193f18670385ca39844657f97a621543af7798c3dfd17b534be
Sep 14 01:46:47.272164 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.272120    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"]
Sep 14 01:46:47.272676 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.272395    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8" podUID="0389bac9-b8c8-4d6b-95dc-7318bc84386d" containerName="maas-api" containerID="cri-o://84ed367b011060b129386030f4f3e37534a61acb8f23cfd1dcea19af30c4073a" gracePeriod=30
Sep 14 01:46:47.482205 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.482178    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:46:47.514491 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.514458    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl" event={"ID":"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc","Type":"ContainerStarted","Data":"8c2ab7f2c696379d193c2e486473b4e110d099431a755ee496c470b278a21a2b"}
Sep 14 01:46:47.514701 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.514508    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl" event={"ID":"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc","Type":"ContainerStarted","Data":"ea03cb6088ffd193f18670385ca39844657f97a621543af7798c3dfd17b534be"}
Sep 14 01:46:47.516543 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.516517    2581 generic.go:356] "Generic (PLEG): container finished" podID="0389bac9-b8c8-4d6b-95dc-7318bc84386d" containerID="84ed367b011060b129386030f4f3e37534a61acb8f23cfd1dcea19af30c4073a" exitCode=0
Sep 14 01:46:47.516697 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.516676    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8" event={"ID":"0389bac9-b8c8-4d6b-95dc-7318bc84386d","Type":"ContainerDied","Data":"84ed367b011060b129386030f4f3e37534a61acb8f23cfd1dcea19af30c4073a"}
Sep 14 01:46:47.533395 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.533293    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl" podStartSLOduration=2.533275924 podStartE2EDuration="2.533275924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:46:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:46:47.532370385 +0000 UTC m=+967.012821202" watchObservedRunningTime="2026-09-14 01:46:47.533275924 +0000 UTC m=+967.013726744"
Sep 14 01:46:47.542627 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.542600    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:47.627710 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.627679    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0389bac9-b8c8-4d6b-95dc-7318bc84386d-openshift-service-ca\") pod \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") "
Sep 14 01:46:47.627710 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.627722    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0389bac9-b8c8-4d6b-95dc-7318bc84386d-metrics-certs\") pod \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") "
Sep 14 01:46:47.627993 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.627755    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbrmc\" (UniqueName: \"kubernetes.io/projected/0389bac9-b8c8-4d6b-95dc-7318bc84386d-kube-api-access-zbrmc\") pod \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") "
Sep 14 01:46:47.627993 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.627838    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0389bac9-b8c8-4d6b-95dc-7318bc84386d-maas-api-tls\") pod \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\" (UID: \"0389bac9-b8c8-4d6b-95dc-7318bc84386d\") "
Sep 14 01:46:47.628110 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.628080    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0389bac9-b8c8-4d6b-95dc-7318bc84386d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0389bac9-b8c8-4d6b-95dc-7318bc84386d" (UID: "0389bac9-b8c8-4d6b-95dc-7318bc84386d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:46:47.629063 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.629037    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0389bac9-b8c8-4d6b-95dc-7318bc84386d-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:46:47.631171 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.631132    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0389bac9-b8c8-4d6b-95dc-7318bc84386d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0389bac9-b8c8-4d6b-95dc-7318bc84386d" (UID: "0389bac9-b8c8-4d6b-95dc-7318bc84386d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:46:47.631306 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.631225    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0389bac9-b8c8-4d6b-95dc-7318bc84386d-kube-api-access-zbrmc" (OuterVolumeSpecName: "kube-api-access-zbrmc") pod "0389bac9-b8c8-4d6b-95dc-7318bc84386d" (UID: "0389bac9-b8c8-4d6b-95dc-7318bc84386d"). InnerVolumeSpecName "kube-api-access-zbrmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:46:47.631401 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.631375    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0389bac9-b8c8-4d6b-95dc-7318bc84386d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0389bac9-b8c8-4d6b-95dc-7318bc84386d" (UID: "0389bac9-b8c8-4d6b-95dc-7318bc84386d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:46:47.729557 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.729523    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0389bac9-b8c8-4d6b-95dc-7318bc84386d-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:46:47.729557 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.729553    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbrmc\" (UniqueName: \"kubernetes.io/projected/0389bac9-b8c8-4d6b-95dc-7318bc84386d-kube-api-access-zbrmc\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:46:47.729557 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:47.729562    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0389bac9-b8c8-4d6b-95dc-7318bc84386d-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:46:48.523736 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:48.523705    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"
Sep 14 01:46:48.523736 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:48.523714    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8" event={"ID":"0389bac9-b8c8-4d6b-95dc-7318bc84386d","Type":"ContainerDied","Data":"1b519562a9af2ded48e369b27782397e96bfdd1d6d2516f5581d19ec53bd33fd"}
Sep 14 01:46:48.524270 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:48.523813    2581 scope.go:117] "RemoveContainer" containerID="84ed367b011060b129386030f4f3e37534a61acb8f23cfd1dcea19af30c4073a"
Sep 14 01:46:48.544220 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:48.544189    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"]
Sep 14 01:46:48.550235 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:48.550212    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e3d76dcd-74f57c6f87-q5pm8"]
Sep 14 01:46:49.057514 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:49.057480    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0389bac9-b8c8-4d6b-95dc-7318bc84386d" path="/var/lib/kubelet/pods/0389bac9-b8c8-4d6b-95dc-7318bc84386d/volumes"
Sep 14 01:46:54.042932 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.042888    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"]
Sep 14 01:46:54.043361 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.043289    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0389bac9-b8c8-4d6b-95dc-7318bc84386d" containerName="maas-api"
Sep 14 01:46:54.043361 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.043302    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0389bac9-b8c8-4d6b-95dc-7318bc84386d" containerName="maas-api"
Sep 14 01:46:54.043459 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.043397    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0389bac9-b8c8-4d6b-95dc-7318bc84386d" containerName="maas-api"
Sep 14 01:46:54.045868 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.045848    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:46:54.047990 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.047963    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-b766dd7c-serving-cert\""
Sep 14 01:46:54.058456 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.058428    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"]
Sep 14 01:46:54.091721 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.091688    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkh58\" (UniqueName: \"kubernetes.io/projected/a14f53fb-e1de-4638-b5d4-49f2fdae4458-kube-api-access-gkh58\") pod \"maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:46:54.091721 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.091722    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a14f53fb-e1de-4638-b5d4-49f2fdae4458-metrics-certs\") pod \"maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:46:54.091959 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.091744    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a14f53fb-e1de-4638-b5d4-49f2fdae4458-openshift-service-ca\") pod \"maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:46:54.091959 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.091856    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a14f53fb-e1de-4638-b5d4-49f2fdae4458-maas-api-tls\") pod \"maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:46:54.192648 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.192604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a14f53fb-e1de-4638-b5d4-49f2fdae4458-maas-api-tls\") pod \"maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:46:54.192832 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.192667    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkh58\" (UniqueName: \"kubernetes.io/projected/a14f53fb-e1de-4638-b5d4-49f2fdae4458-kube-api-access-gkh58\") pod \"maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:46:54.192832 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.192694    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a14f53fb-e1de-4638-b5d4-49f2fdae4458-metrics-certs\") pod \"maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:46:54.192832 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.192722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a14f53fb-e1de-4638-b5d4-49f2fdae4458-openshift-service-ca\") pod \"maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:46:54.193477 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.193455    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a14f53fb-e1de-4638-b5d4-49f2fdae4458-openshift-service-ca\") pod \"maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:46:54.195245 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.195228    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a14f53fb-e1de-4638-b5d4-49f2fdae4458-maas-api-tls\") pod \"maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:46:54.195402 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.195380    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a14f53fb-e1de-4638-b5d4-49f2fdae4458-metrics-certs\") pod \"maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:46:54.201751 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.201717    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkh58\" (UniqueName: \"kubernetes.io/projected/a14f53fb-e1de-4638-b5d4-49f2fdae4458-kube-api-access-gkh58\") pod \"maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:46:54.359230 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.359144    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:46:54.633693 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:54.633662    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"]
Sep 14 01:46:54.635466 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:46:54.635437    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda14f53fb_e1de_4638_b5d4_49f2fdae4458.slice/crio-7f78ffdaca2bb440c4f1a4fd93ea70eae59898af36620a02ab8aa57986fe722f WatchSource:0}: Error finding container 7f78ffdaca2bb440c4f1a4fd93ea70eae59898af36620a02ab8aa57986fe722f: Status 404 returned error can't find the container with id 7f78ffdaca2bb440c4f1a4fd93ea70eae59898af36620a02ab8aa57986fe722f
Sep 14 01:46:55.571449 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:55.571410    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps" event={"ID":"a14f53fb-e1de-4638-b5d4-49f2fdae4458","Type":"ContainerStarted","Data":"4b912be9bf345e812e29145c9f0a3b6c509d04a97b66e9b9d953b029a0e7d2d3"}
Sep 14 01:46:55.571967 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:55.571467    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps" event={"ID":"a14f53fb-e1de-4638-b5d4-49f2fdae4458","Type":"ContainerStarted","Data":"7f78ffdaca2bb440c4f1a4fd93ea70eae59898af36620a02ab8aa57986fe722f"}
Sep 14 01:46:55.571967 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:55.571524    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:46:55.587860 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:55.587807    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps" podStartSLOduration=1.26085339 podStartE2EDuration="1.587791164s" totalImagesPullingTime="326.937774ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:46:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:46:55.585488578 +0000 UTC m=+975.065939400" watchObservedRunningTime="2026-09-14 01:46:55.587791164 +0000 UTC m=+975.068241984"
Sep 14 01:46:55.663954 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:55.663894    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:46:56.309281 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:56.309239    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:56.309479 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:56.309307    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:56.323032 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:56.323002    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:46:56.588651 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:46:56.588576    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:47:01.583404 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:01.583376    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:47:04.498844 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.498762    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"]
Sep 14 01:47:04.502831 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.502812    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.505467 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.505445    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-8e5935\"/\"openshift-service-ca.crt\""
Sep 14 01:47:04.505601 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.505489    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-8e5935\"/\"default-dockercfg-j965m\""
Sep 14 01:47:04.505601 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.505445    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-8e5935\"/\"simulator-w4-8e5935-kserve-self-signed-certs\""
Sep 14 01:47:04.505601 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.505448    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-8e5935\"/\"kube-root-ca.crt\""
Sep 14 01:47:04.511402 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.511380    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"]
Sep 14 01:47:04.581178 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.581142    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-model-cache\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.581344 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.581191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-dshm\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.581344 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.581212    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db632892-fbe4-4609-a892-739324cf50f7-tls-certs\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.581344 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.581314    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bpf9\" (UniqueName: \"kubernetes.io/projected/db632892-fbe4-4609-a892-739324cf50f7-kube-api-access-5bpf9\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.581480 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.581371    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-tmp-dir\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.581480 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.581423    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-home\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.682224 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.682181    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-tmp-dir\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.682421 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.682259    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-home\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.682421 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.682302    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-model-cache\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.682421 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.682346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-dshm\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.682421 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.682366    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db632892-fbe4-4609-a892-739324cf50f7-tls-certs\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.682645 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.682421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bpf9\" (UniqueName: \"kubernetes.io/projected/db632892-fbe4-4609-a892-739324cf50f7-kube-api-access-5bpf9\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.682708 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.682665    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-tmp-dir\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.682768 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.682729    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-home\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.682819 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.682753    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-model-cache\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.684711 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.684680    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-dshm\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.684981 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.684965    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db632892-fbe4-4609-a892-739324cf50f7-tls-certs\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.689485 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.689460    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bpf9\" (UniqueName: \"kubernetes.io/projected/db632892-fbe4-4609-a892-739324cf50f7-kube-api-access-5bpf9\") pod \"simulator-w4-8e5935-kserve-5fb756b74-q5ct8\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") " pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:04.814530 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:04.814438    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:05.092820 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:05.092793    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"]
Sep 14 01:47:05.094672 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:47:05.094645    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb632892_fbe4_4609_a892_739324cf50f7.slice/crio-4babc8891ea31cd2638fcc2dc04ad04f10bfefb4b46a52412c60028c8d1a9a77 WatchSource:0}: Error finding container 4babc8891ea31cd2638fcc2dc04ad04f10bfefb4b46a52412c60028c8d1a9a77: Status 404 returned error can't find the container with id 4babc8891ea31cd2638fcc2dc04ad04f10bfefb4b46a52412c60028c8d1a9a77
Sep 14 01:47:05.629465 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:05.629429    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8" event={"ID":"db632892-fbe4-4609-a892-739324cf50f7","Type":"ContainerStarted","Data":"3e64c52600dc9111e56c3985cae42f74852f54c42375ae50c92bb3265aad9dd3"}
Sep 14 01:47:05.629903 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:05.629473    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8" event={"ID":"db632892-fbe4-4609-a892-739324cf50f7","Type":"ContainerStarted","Data":"4babc8891ea31cd2638fcc2dc04ad04f10bfefb4b46a52412c60028c8d1a9a77"}
Sep 14 01:47:05.646057 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:05.646001    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8" podStartSLOduration=1.645980894 podStartE2EDuration="1.645980894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:47:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:47:05.643186175 +0000 UTC m=+985.123636994" watchObservedRunningTime="2026-09-14 01:47:05.645980894 +0000 UTC m=+985.126431715"
Sep 14 01:47:13.157382 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:13.157341    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:47:14.814878 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:14.814834    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:14.815287 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:14.814891    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:14.827676 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:14.827649    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:15.695863 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:15.695833    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 01:47:21.707047 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:21.706875    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"]
Sep 14 01:47:21.707625 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:21.707187    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps" podUID="a14f53fb-e1de-4638-b5d4-49f2fdae4458" containerName="maas-api" containerID="cri-o://4b912be9bf345e812e29145c9f0a3b6c509d04a97b66e9b9d953b029a0e7d2d3" gracePeriod=30
Sep 14 01:47:21.962737 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:21.962665    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:47:22.045553 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.045520    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a14f53fb-e1de-4638-b5d4-49f2fdae4458-openshift-service-ca\") pod \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") "
Sep 14 01:47:22.045753 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.045587    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a14f53fb-e1de-4638-b5d4-49f2fdae4458-maas-api-tls\") pod \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") "
Sep 14 01:47:22.045753 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.045629    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a14f53fb-e1de-4638-b5d4-49f2fdae4458-metrics-certs\") pod \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") "
Sep 14 01:47:22.045753 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.045668    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkh58\" (UniqueName: \"kubernetes.io/projected/a14f53fb-e1de-4638-b5d4-49f2fdae4458-kube-api-access-gkh58\") pod \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\" (UID: \"a14f53fb-e1de-4638-b5d4-49f2fdae4458\") "
Sep 14 01:47:22.045951 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.045852    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a14f53fb-e1de-4638-b5d4-49f2fdae4458-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a14f53fb-e1de-4638-b5d4-49f2fdae4458" (UID: "a14f53fb-e1de-4638-b5d4-49f2fdae4458"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:47:22.046261 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.046225    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a14f53fb-e1de-4638-b5d4-49f2fdae4458-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:47:22.049411 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.049381    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a14f53fb-e1de-4638-b5d4-49f2fdae4458-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a14f53fb-e1de-4638-b5d4-49f2fdae4458" (UID: "a14f53fb-e1de-4638-b5d4-49f2fdae4458"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:47:22.049411 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.049397    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a14f53fb-e1de-4638-b5d4-49f2fdae4458-kube-api-access-gkh58" (OuterVolumeSpecName: "kube-api-access-gkh58") pod "a14f53fb-e1de-4638-b5d4-49f2fdae4458" (UID: "a14f53fb-e1de-4638-b5d4-49f2fdae4458"). InnerVolumeSpecName "kube-api-access-gkh58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:47:22.049607 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.049412    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a14f53fb-e1de-4638-b5d4-49f2fdae4458-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a14f53fb-e1de-4638-b5d4-49f2fdae4458" (UID: "a14f53fb-e1de-4638-b5d4-49f2fdae4458"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:47:22.147128 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.147093    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a14f53fb-e1de-4638-b5d4-49f2fdae4458-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:47:22.147128 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.147121    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a14f53fb-e1de-4638-b5d4-49f2fdae4458-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:47:22.147128 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.147130    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkh58\" (UniqueName: \"kubernetes.io/projected/a14f53fb-e1de-4638-b5d4-49f2fdae4458-kube-api-access-gkh58\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:47:22.733232 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.733194    2581 generic.go:356] "Generic (PLEG): container finished" podID="a14f53fb-e1de-4638-b5d4-49f2fdae4458" containerID="4b912be9bf345e812e29145c9f0a3b6c509d04a97b66e9b9d953b029a0e7d2d3" exitCode=0
Sep 14 01:47:22.733653 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.733239    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"
Sep 14 01:47:22.733653 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.733271    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps" event={"ID":"a14f53fb-e1de-4638-b5d4-49f2fdae4458","Type":"ContainerDied","Data":"4b912be9bf345e812e29145c9f0a3b6c509d04a97b66e9b9d953b029a0e7d2d3"}
Sep 14 01:47:22.733653 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.733325    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps" event={"ID":"a14f53fb-e1de-4638-b5d4-49f2fdae4458","Type":"ContainerDied","Data":"7f78ffdaca2bb440c4f1a4fd93ea70eae59898af36620a02ab8aa57986fe722f"}
Sep 14 01:47:22.733653 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.733350    2581 scope.go:117] "RemoveContainer" containerID="4b912be9bf345e812e29145c9f0a3b6c509d04a97b66e9b9d953b029a0e7d2d3"
Sep 14 01:47:22.743148 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.743130    2581 scope.go:117] "RemoveContainer" containerID="4b912be9bf345e812e29145c9f0a3b6c509d04a97b66e9b9d953b029a0e7d2d3"
Sep 14 01:47:22.743502 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:47:22.743482    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b912be9bf345e812e29145c9f0a3b6c509d04a97b66e9b9d953b029a0e7d2d3\": container with ID starting with 4b912be9bf345e812e29145c9f0a3b6c509d04a97b66e9b9d953b029a0e7d2d3 not found: ID does not exist" containerID="4b912be9bf345e812e29145c9f0a3b6c509d04a97b66e9b9d953b029a0e7d2d3"
Sep 14 01:47:22.743580 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.743514    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b912be9bf345e812e29145c9f0a3b6c509d04a97b66e9b9d953b029a0e7d2d3"} err="failed to get container status \"4b912be9bf345e812e29145c9f0a3b6c509d04a97b66e9b9d953b029a0e7d2d3\": rpc error: code = NotFound desc = could not find container \"4b912be9bf345e812e29145c9f0a3b6c509d04a97b66e9b9d953b029a0e7d2d3\": container with ID starting with 4b912be9bf345e812e29145c9f0a3b6c509d04a97b66e9b9d953b029a0e7d2d3 not found: ID does not exist"
Sep 14 01:47:22.755679 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.755651    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"]
Sep 14 01:47:22.763579 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:22.763551    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b766dd7c-55fdf75fb7-9n5ps"]
Sep 14 01:47:23.055111 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:23.055027    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a14f53fb-e1de-4638-b5d4-49f2fdae4458" path="/var/lib/kubelet/pods/a14f53fb-e1de-4638-b5d4-49f2fdae4458/volumes"
Sep 14 01:47:27.009425 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.009385    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"]
Sep 14 01:47:27.010061 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.010043    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a14f53fb-e1de-4638-b5d4-49f2fdae4458" containerName="maas-api"
Sep 14 01:47:27.010118 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.010064    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a14f53fb-e1de-4638-b5d4-49f2fdae4458" containerName="maas-api"
Sep 14 01:47:27.010179 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.010168    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a14f53fb-e1de-4638-b5d4-49f2fdae4458" containerName="maas-api"
Sep 14 01:47:27.015266 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.015245    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:27.017117 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.017096    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-91e5cec3-serving-cert\""
Sep 14 01:47:27.024900 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.024872    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"]
Sep 14 01:47:27.101977 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.101943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67da7330-56a4-4363-b51f-5ca98f3d9a88-metrics-certs\") pod \"maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:27.102183 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.101993    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pkx9\" (UniqueName: \"kubernetes.io/projected/67da7330-56a4-4363-b51f-5ca98f3d9a88-kube-api-access-4pkx9\") pod \"maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:27.102183 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.102101    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67da7330-56a4-4363-b51f-5ca98f3d9a88-maas-api-tls\") pod \"maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:27.102183 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.102137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67da7330-56a4-4363-b51f-5ca98f3d9a88-openshift-service-ca\") pod \"maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:27.202812 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.202780    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67da7330-56a4-4363-b51f-5ca98f3d9a88-metrics-certs\") pod \"maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:27.203018 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.202832    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pkx9\" (UniqueName: \"kubernetes.io/projected/67da7330-56a4-4363-b51f-5ca98f3d9a88-kube-api-access-4pkx9\") pod \"maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:27.203018 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.202879    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67da7330-56a4-4363-b51f-5ca98f3d9a88-maas-api-tls\") pod \"maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:27.203018 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.202897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67da7330-56a4-4363-b51f-5ca98f3d9a88-openshift-service-ca\") pod \"maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:27.203196 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:47:27.203130    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-serving-cert: secret "maas-api-e2e-ait-91e5cec3-serving-cert" not found
Sep 14 01:47:27.203196 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:47:27.203194    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67da7330-56a4-4363-b51f-5ca98f3d9a88-maas-api-tls podName:67da7330-56a4-4363-b51f-5ca98f3d9a88 nodeName:}" failed. No retries permitted until 2026-09-14 01:47:27.703179112 +0000 UTC m=+1007.183629911 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/67da7330-56a4-4363-b51f-5ca98f3d9a88-maas-api-tls") pod "maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89" (UID: "67da7330-56a4-4363-b51f-5ca98f3d9a88") : secret "maas-api-e2e-ait-91e5cec3-serving-cert" not found
Sep 14 01:47:27.203695 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.203668    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67da7330-56a4-4363-b51f-5ca98f3d9a88-openshift-service-ca\") pod \"maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:27.205594 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.205571    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67da7330-56a4-4363-b51f-5ca98f3d9a88-metrics-certs\") pod \"maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:27.212409 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.212380    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pkx9\" (UniqueName: \"kubernetes.io/projected/67da7330-56a4-4363-b51f-5ca98f3d9a88-kube-api-access-4pkx9\") pod \"maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:27.707630 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.707521    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67da7330-56a4-4363-b51f-5ca98f3d9a88-maas-api-tls\") pod \"maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:27.709986 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.709966    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67da7330-56a4-4363-b51f-5ca98f3d9a88-maas-api-tls\") pod \"maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:27.929868 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:27.929829    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:28.058826 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:28.058790    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:47:28.203318 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:28.203288    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"]
Sep 14 01:47:28.204813 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:47:28.204787    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67da7330_56a4_4363_b51f_5ca98f3d9a88.slice/crio-fa27833ea9adc979233956b828ff9674d37b77905f0798d8c9cd19c9c5a8d9b5 WatchSource:0}: Error finding container fa27833ea9adc979233956b828ff9674d37b77905f0798d8c9cd19c9c5a8d9b5: Status 404 returned error can't find the container with id fa27833ea9adc979233956b828ff9674d37b77905f0798d8c9cd19c9c5a8d9b5
Sep 14 01:47:28.770146 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:28.770112    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89" event={"ID":"67da7330-56a4-4363-b51f-5ca98f3d9a88","Type":"ContainerStarted","Data":"c615e53247df33a63fa4777ce4f2ff73d146c952fb87df723184adcaeb13c466"}
Sep 14 01:47:28.770340 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:28.770175    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89" event={"ID":"67da7330-56a4-4363-b51f-5ca98f3d9a88","Type":"ContainerStarted","Data":"fa27833ea9adc979233956b828ff9674d37b77905f0798d8c9cd19c9c5a8d9b5"}
Sep 14 01:47:28.770340 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:28.770207    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:28.786211 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:28.786159    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89" podStartSLOduration=2.349642777 podStartE2EDuration="2.786123215s" totalImagesPullingTime="436.480438ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:47:28.784673394 +0000 UTC m=+1008.265124238" watchObservedRunningTime="2026-09-14 01:47:28.786123215 +0000 UTC m=+1008.266574035"
Sep 14 01:47:34.399624 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.399582    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"]
Sep 14 01:47:34.403130 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.403110    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.405790 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.405750    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-bc2e37\"/\"autores-model-bc2e37-kserve-self-signed-certs\""
Sep 14 01:47:34.405790 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.405751    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-bc2e37\"/\"openshift-service-ca.crt\""
Sep 14 01:47:34.405790 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.405771    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-bc2e37\"/\"default-dockercfg-jv4fp\""
Sep 14 01:47:34.406111 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.405804    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-bc2e37\"/\"kube-root-ca.crt\""
Sep 14 01:47:34.412603 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.412575    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"]
Sep 14 01:47:34.573975 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.573940    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-tmp-dir\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.574136 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.573993    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-dshm\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.574136 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.574022    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-home\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.574136 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.574076    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjbpw\" (UniqueName: \"kubernetes.io/projected/628b4a0b-9431-4434-8bd4-25c602fccda0-kube-api-access-tjbpw\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.574136 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.574104    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628b4a0b-9431-4434-8bd4-25c602fccda0-tls-certs\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.574136 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.574125    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-model-cache\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.675324 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.675231    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-tmp-dir\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.675324 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.675280    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-dshm\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.675324 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.675298    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-home\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.675613 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.675343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjbpw\" (UniqueName: \"kubernetes.io/projected/628b4a0b-9431-4434-8bd4-25c602fccda0-kube-api-access-tjbpw\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.675613 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.675361    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628b4a0b-9431-4434-8bd4-25c602fccda0-tls-certs\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.675613 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.675379    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-model-cache\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.675770 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.675703    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-tmp-dir\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.675816 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.675768    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-home\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.675850 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.675829    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-model-cache\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.677903 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.677878    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-dshm\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.678098 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.677976    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628b4a0b-9431-4434-8bd4-25c602fccda0-tls-certs\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.682759 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.682740    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjbpw\" (UniqueName: \"kubernetes.io/projected/628b4a0b-9431-4434-8bd4-25c602fccda0-kube-api-access-tjbpw\") pod \"autores-model-bc2e37-kserve-5dbccdbff5-qcp9t\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") " pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.715281 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.715251    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:34.783066 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.782951    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:34.989870 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:34.989842    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"]
Sep 14 01:47:34.990434 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:47:34.990406    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod628b4a0b_9431_4434_8bd4_25c602fccda0.slice/crio-36d2cad79b4f8a2dd4f730b6bfbb13a6e6d3caf317c73458246dedfa2fb8f3ab WatchSource:0}: Error finding container 36d2cad79b4f8a2dd4f730b6bfbb13a6e6d3caf317c73458246dedfa2fb8f3ab: Status 404 returned error can't find the container with id 36d2cad79b4f8a2dd4f730b6bfbb13a6e6d3caf317c73458246dedfa2fb8f3ab
Sep 14 01:47:35.813639 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:35.813607    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t" event={"ID":"628b4a0b-9431-4434-8bd4-25c602fccda0","Type":"ContainerStarted","Data":"87d82c230be82a6236a490df93bd3fbd19d8c2e12eb3989eb5e1b05b3809de0b"}
Sep 14 01:47:35.814096 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:35.813651    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t" event={"ID":"628b4a0b-9431-4434-8bd4-25c602fccda0","Type":"ContainerStarted","Data":"36d2cad79b4f8a2dd4f730b6bfbb13a6e6d3caf317c73458246dedfa2fb8f3ab"}
Sep 14 01:47:35.828185 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:35.828136    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t" podStartSLOduration=1.828119734 podStartE2EDuration="1.828119734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:47:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:47:35.827383274 +0000 UTC m=+1015.307834095" watchObservedRunningTime="2026-09-14 01:47:35.828119734 +0000 UTC m=+1015.308570557"
Sep 14 01:47:44.715795 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:44.715756    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:44.716267 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:44.715823    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:44.728477 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:44.728451    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:44.878036 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:44.878008    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:47:55.271728 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:55.271686    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:47:57.057439 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.057405    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"]
Sep 14 01:47:57.057882 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.057731    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89" podUID="67da7330-56a4-4363-b51f-5ca98f3d9a88" containerName="maas-api" containerID="cri-o://c615e53247df33a63fa4777ce4f2ff73d146c952fb87df723184adcaeb13c466" gracePeriod=30
Sep 14 01:47:57.315246 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.315180    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:57.354288 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.354244    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67da7330-56a4-4363-b51f-5ca98f3d9a88-maas-api-tls\") pod \"67da7330-56a4-4363-b51f-5ca98f3d9a88\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") "
Sep 14 01:47:57.354288 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.354284    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67da7330-56a4-4363-b51f-5ca98f3d9a88-openshift-service-ca\") pod \"67da7330-56a4-4363-b51f-5ca98f3d9a88\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") "
Sep 14 01:47:57.354554 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.354305    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pkx9\" (UniqueName: \"kubernetes.io/projected/67da7330-56a4-4363-b51f-5ca98f3d9a88-kube-api-access-4pkx9\") pod \"67da7330-56a4-4363-b51f-5ca98f3d9a88\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") "
Sep 14 01:47:57.354554 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.354350    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67da7330-56a4-4363-b51f-5ca98f3d9a88-metrics-certs\") pod \"67da7330-56a4-4363-b51f-5ca98f3d9a88\" (UID: \"67da7330-56a4-4363-b51f-5ca98f3d9a88\") "
Sep 14 01:47:57.354685 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.354647    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67da7330-56a4-4363-b51f-5ca98f3d9a88-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "67da7330-56a4-4363-b51f-5ca98f3d9a88" (UID: "67da7330-56a4-4363-b51f-5ca98f3d9a88"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:47:57.356582 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.356548    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67da7330-56a4-4363-b51f-5ca98f3d9a88-kube-api-access-4pkx9" (OuterVolumeSpecName: "kube-api-access-4pkx9") pod "67da7330-56a4-4363-b51f-5ca98f3d9a88" (UID: "67da7330-56a4-4363-b51f-5ca98f3d9a88"). InnerVolumeSpecName "kube-api-access-4pkx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:47:57.356953 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.356610    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67da7330-56a4-4363-b51f-5ca98f3d9a88-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "67da7330-56a4-4363-b51f-5ca98f3d9a88" (UID: "67da7330-56a4-4363-b51f-5ca98f3d9a88"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:47:57.356953 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.356628    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67da7330-56a4-4363-b51f-5ca98f3d9a88-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "67da7330-56a4-4363-b51f-5ca98f3d9a88" (UID: "67da7330-56a4-4363-b51f-5ca98f3d9a88"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:47:57.455837 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.455797    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67da7330-56a4-4363-b51f-5ca98f3d9a88-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:47:57.455837 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.455830    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pkx9\" (UniqueName: \"kubernetes.io/projected/67da7330-56a4-4363-b51f-5ca98f3d9a88-kube-api-access-4pkx9\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:47:57.455837 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.455841    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67da7330-56a4-4363-b51f-5ca98f3d9a88-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:47:57.455837 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.455850    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67da7330-56a4-4363-b51f-5ca98f3d9a88-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:47:57.941249 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.941213    2581 generic.go:356] "Generic (PLEG): container finished" podID="67da7330-56a4-4363-b51f-5ca98f3d9a88" containerID="c615e53247df33a63fa4777ce4f2ff73d146c952fb87df723184adcaeb13c466" exitCode=0
Sep 14 01:47:57.941465 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.941287    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"
Sep 14 01:47:57.941465 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.941301    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89" event={"ID":"67da7330-56a4-4363-b51f-5ca98f3d9a88","Type":"ContainerDied","Data":"c615e53247df33a63fa4777ce4f2ff73d146c952fb87df723184adcaeb13c466"}
Sep 14 01:47:57.941465 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.941389    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89" event={"ID":"67da7330-56a4-4363-b51f-5ca98f3d9a88","Type":"ContainerDied","Data":"fa27833ea9adc979233956b828ff9674d37b77905f0798d8c9cd19c9c5a8d9b5"}
Sep 14 01:47:57.941465 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.941413    2581 scope.go:117] "RemoveContainer" containerID="c615e53247df33a63fa4777ce4f2ff73d146c952fb87df723184adcaeb13c466"
Sep 14 01:47:57.951548 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.951529    2581 scope.go:117] "RemoveContainer" containerID="c615e53247df33a63fa4777ce4f2ff73d146c952fb87df723184adcaeb13c466"
Sep 14 01:47:57.951932 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:47:57.951895    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c615e53247df33a63fa4777ce4f2ff73d146c952fb87df723184adcaeb13c466\": container with ID starting with c615e53247df33a63fa4777ce4f2ff73d146c952fb87df723184adcaeb13c466 not found: ID does not exist" containerID="c615e53247df33a63fa4777ce4f2ff73d146c952fb87df723184adcaeb13c466"
Sep 14 01:47:57.951998 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.951946    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c615e53247df33a63fa4777ce4f2ff73d146c952fb87df723184adcaeb13c466"} err="failed to get container status \"c615e53247df33a63fa4777ce4f2ff73d146c952fb87df723184adcaeb13c466\": rpc error: code = NotFound desc = could not find container \"c615e53247df33a63fa4777ce4f2ff73d146c952fb87df723184adcaeb13c466\": container with ID starting with c615e53247df33a63fa4777ce4f2ff73d146c952fb87df723184adcaeb13c466 not found: ID does not exist"
Sep 14 01:47:57.961962 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.961939    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"]
Sep 14 01:47:57.969532 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:57.969510    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-91e5cec3-6c4ddcd4d9-4zw89"]
Sep 14 01:47:59.056172 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:47:59.056132    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67da7330-56a4-4363-b51f-5ca98f3d9a88" path="/var/lib/kubelet/pods/67da7330-56a4-4363-b51f-5ca98f3d9a88/volumes"
Sep 14 01:48:05.117902 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.117866    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"]
Sep 14 01:48:05.118379 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.118363    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67da7330-56a4-4363-b51f-5ca98f3d9a88" containerName="maas-api"
Sep 14 01:48:05.118379 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.118377    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="67da7330-56a4-4363-b51f-5ca98f3d9a88" containerName="maas-api"
Sep 14 01:48:05.118492 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.118480    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="67da7330-56a4-4363-b51f-5ca98f3d9a88" containerName="maas-api"
Sep 14 01:48:05.123049 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.123017    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:05.125200 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.125174    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-b351ffdb-serving-cert\""
Sep 14 01:48:05.133836 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.133807    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"]
Sep 14 01:48:05.228019 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.227983    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks525\" (UniqueName: \"kubernetes.io/projected/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-kube-api-access-ks525\") pod \"maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:05.228219 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.228035    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-maas-api-tls\") pod \"maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:05.228219 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.228060    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-openshift-service-ca\") pod \"maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:05.228219 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.228150    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-metrics-certs\") pod \"maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:05.329485 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.329441    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-maas-api-tls\") pod \"maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:05.329485 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.329484    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-openshift-service-ca\") pod \"maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:05.329758 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.329564    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-metrics-certs\") pod \"maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:05.329758 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:48:05.329590    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-serving-cert: secret "maas-api-e2e-derive-b351ffdb-serving-cert" not found
Sep 14 01:48:05.329758 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.329661    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks525\" (UniqueName: \"kubernetes.io/projected/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-kube-api-access-ks525\") pod \"maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:05.329758 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:48:05.329680    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-maas-api-tls podName:292cfd39-635f-4dfc-9e4c-176b77ad5ea1 nodeName:}" failed. No retries permitted until 2026-09-14 01:48:05.829655295 +0000 UTC m=+1045.310106096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-maas-api-tls") pod "maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff" (UID: "292cfd39-635f-4dfc-9e4c-176b77ad5ea1") : secret "maas-api-e2e-derive-b351ffdb-serving-cert" not found
Sep 14 01:48:05.330307 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.330288    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-openshift-service-ca\") pod \"maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:05.332157 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.332136    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-metrics-certs\") pod \"maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:05.336887 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.336864    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks525\" (UniqueName: \"kubernetes.io/projected/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-kube-api-access-ks525\") pod \"maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:05.834969 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.834892    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-maas-api-tls\") pod \"maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:05.837494 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:05.837465    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-maas-api-tls\") pod \"maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:06.038813 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:06.038782    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:06.314934 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:06.314892    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"]
Sep 14 01:48:06.316080 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:48:06.316053    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod292cfd39_635f_4dfc_9e4c_176b77ad5ea1.slice/crio-c80a549abe8f938e4422b82815b059f6e91bc73b0562440636b03183f8c5f508 WatchSource:0}: Error finding container c80a549abe8f938e4422b82815b059f6e91bc73b0562440636b03183f8c5f508: Status 404 returned error can't find the container with id c80a549abe8f938e4422b82815b059f6e91bc73b0562440636b03183f8c5f508
Sep 14 01:48:06.994440 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:06.994398    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff" event={"ID":"292cfd39-635f-4dfc-9e4c-176b77ad5ea1","Type":"ContainerStarted","Data":"44c5e2068be7e4d822944db6907e25c3199be8612bdc54c9b515b2e23fbf7c2e"}
Sep 14 01:48:06.994619 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:06.994457    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff" event={"ID":"292cfd39-635f-4dfc-9e4c-176b77ad5ea1","Type":"ContainerStarted","Data":"c80a549abe8f938e4422b82815b059f6e91bc73b0562440636b03183f8c5f508"}
Sep 14 01:48:06.994619 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:06.994522    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:07.010002 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:07.009958    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff" podStartSLOduration=1.631445733 podStartE2EDuration="2.00994205s" totalImagesPullingTime="378.496317ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:48:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:48:07.009039069 +0000 UTC m=+1046.489489890" watchObservedRunningTime="2026-09-14 01:48:07.00994205 +0000 UTC m=+1046.490392868"
Sep 14 01:48:12.896190 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:12.896153    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"]
Sep 14 01:48:12.899870 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:12.899852    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:12.901943 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:12.901900    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a36bf3\"/\"kube-root-ca.crt\""
Sep 14 01:48:12.902095 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:12.901961    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a36bf3\"/\"openshift-service-ca.crt\""
Sep 14 01:48:12.902524 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:12.902503    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a36bf3\"/\"test-model-a36bf3-kserve-self-signed-certs\""
Sep 14 01:48:12.902641 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:12.902505    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a36bf3\"/\"default-dockercfg-72q47\""
Sep 14 01:48:12.906980 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:12.906954    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"]
Sep 14 01:48:13.004256 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.004218    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzwq9\" (UniqueName: \"kubernetes.io/projected/d72a2797-4017-460f-83f6-6b8444e5e9df-kube-api-access-nzwq9\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.004435 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.004265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d72a2797-4017-460f-83f6-6b8444e5e9df-tls-certs\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.004435 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.004322    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-model-cache\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.004435 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.004397    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-home\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.004579 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.004475    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-tmp-dir\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.004579 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.004524    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-dshm\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.007411 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.007393    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:13.105753 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.105723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d72a2797-4017-460f-83f6-6b8444e5e9df-tls-certs\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.105959 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.105783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-model-cache\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.105959 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.105883    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-home\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.106096 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.105956    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-tmp-dir\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.106096 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.105984    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-dshm\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.106096 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.106050    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzwq9\" (UniqueName: \"kubernetes.io/projected/d72a2797-4017-460f-83f6-6b8444e5e9df-kube-api-access-nzwq9\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.106398 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.106373    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-model-cache\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.106512 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.106422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-home\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.106718 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.106695    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-tmp-dir\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.108450 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.108428    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-dshm\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.108551 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.108427    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d72a2797-4017-460f-83f6-6b8444e5e9df-tls-certs\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.113111 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.113085    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzwq9\" (UniqueName: \"kubernetes.io/projected/d72a2797-4017-460f-83f6-6b8444e5e9df-kube-api-access-nzwq9\") pod \"test-model-a36bf3-kserve-555dd98c47-85pv5\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") " pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.210183 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.210146    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:13.484665 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:13.484639    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"]
Sep 14 01:48:13.487006 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:48:13.486975    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd72a2797_4017_460f_83f6_6b8444e5e9df.slice/crio-f18754fe89f305098e8e0749df9216d95c1a086ba6e2dbe5f78decdab5ef092f WatchSource:0}: Error finding container f18754fe89f305098e8e0749df9216d95c1a086ba6e2dbe5f78decdab5ef092f: Status 404 returned error can't find the container with id f18754fe89f305098e8e0749df9216d95c1a086ba6e2dbe5f78decdab5ef092f
Sep 14 01:48:14.047054 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:14.047016    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5" event={"ID":"d72a2797-4017-460f-83f6-6b8444e5e9df","Type":"ContainerStarted","Data":"e7608ca28122f8bdfd5fe116416252479319b33f2a54af2129ec16daa040c46e"}
Sep 14 01:48:14.047054 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:14.047065    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5" event={"ID":"d72a2797-4017-460f-83f6-6b8444e5e9df","Type":"ContainerStarted","Data":"f18754fe89f305098e8e0749df9216d95c1a086ba6e2dbe5f78decdab5ef092f"}
Sep 14 01:48:14.061272 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:14.061219    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5" podStartSLOduration=2.061201099 podStartE2EDuration="2.061201099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:48:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:48:14.06047759 +0000 UTC m=+1053.540928420" watchObservedRunningTime="2026-09-14 01:48:14.061201099 +0000 UTC m=+1053.541651920"
Sep 14 01:48:15.468228 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:15.468190    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:48:16.122538 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:16.122497    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"]
Sep 14 01:48:16.123087 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:16.123053    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7" podUID="9909d893-a3d9-45ce-bd57-3a1ed91a344d" containerName="maas-api" containerID="cri-o://cae769fd0b5c243f84fefad95a0a51c412ddc7adc472856f313593b7528e65b6" gracePeriod=30
Sep 14 01:48:16.386456 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:16.386388    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:48:16.542162 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:16.542126    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9909d893-a3d9-45ce-bd57-3a1ed91a344d-metrics-certs\") pod \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") "
Sep 14 01:48:16.542637 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:16.542189    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcwgl\" (UniqueName: \"kubernetes.io/projected/9909d893-a3d9-45ce-bd57-3a1ed91a344d-kube-api-access-rcwgl\") pod \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") "
Sep 14 01:48:16.542637 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:16.542238    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9909d893-a3d9-45ce-bd57-3a1ed91a344d-openshift-service-ca\") pod \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") "
Sep 14 01:48:16.542637 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:16.542264    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9909d893-a3d9-45ce-bd57-3a1ed91a344d-maas-api-tls\") pod \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\" (UID: \"9909d893-a3d9-45ce-bd57-3a1ed91a344d\") "
Sep 14 01:48:16.542637 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:16.542625    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9909d893-a3d9-45ce-bd57-3a1ed91a344d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9909d893-a3d9-45ce-bd57-3a1ed91a344d" (UID: "9909d893-a3d9-45ce-bd57-3a1ed91a344d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:48:16.544601 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:16.544568    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9909d893-a3d9-45ce-bd57-3a1ed91a344d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9909d893-a3d9-45ce-bd57-3a1ed91a344d" (UID: "9909d893-a3d9-45ce-bd57-3a1ed91a344d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:48:16.544739 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:16.544704    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9909d893-a3d9-45ce-bd57-3a1ed91a344d-kube-api-access-rcwgl" (OuterVolumeSpecName: "kube-api-access-rcwgl") pod "9909d893-a3d9-45ce-bd57-3a1ed91a344d" (UID: "9909d893-a3d9-45ce-bd57-3a1ed91a344d"). InnerVolumeSpecName "kube-api-access-rcwgl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:48:16.544845 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:16.544826    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9909d893-a3d9-45ce-bd57-3a1ed91a344d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9909d893-a3d9-45ce-bd57-3a1ed91a344d" (UID: "9909d893-a3d9-45ce-bd57-3a1ed91a344d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:48:16.643397 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:16.643305    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9909d893-a3d9-45ce-bd57-3a1ed91a344d-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:48:16.643397 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:16.643334    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcwgl\" (UniqueName: \"kubernetes.io/projected/9909d893-a3d9-45ce-bd57-3a1ed91a344d-kube-api-access-rcwgl\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:48:16.643397 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:16.643345    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9909d893-a3d9-45ce-bd57-3a1ed91a344d-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:48:16.643397 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:16.643353    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9909d893-a3d9-45ce-bd57-3a1ed91a344d-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:48:17.066008 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:17.065976    2581 generic.go:356] "Generic (PLEG): container finished" podID="9909d893-a3d9-45ce-bd57-3a1ed91a344d" containerID="cae769fd0b5c243f84fefad95a0a51c412ddc7adc472856f313593b7528e65b6" exitCode=0
Sep 14 01:48:17.066139 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:17.066025    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"
Sep 14 01:48:17.066139 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:17.066059    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7" event={"ID":"9909d893-a3d9-45ce-bd57-3a1ed91a344d","Type":"ContainerDied","Data":"cae769fd0b5c243f84fefad95a0a51c412ddc7adc472856f313593b7528e65b6"}
Sep 14 01:48:17.066139 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:17.066116    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7" event={"ID":"9909d893-a3d9-45ce-bd57-3a1ed91a344d","Type":"ContainerDied","Data":"65e8af485c76104e7659399ffd8e3ff537871ff15e6055730c132cb5ac0ff151"}
Sep 14 01:48:17.066267 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:17.066142    2581 scope.go:117] "RemoveContainer" containerID="cae769fd0b5c243f84fefad95a0a51c412ddc7adc472856f313593b7528e65b6"
Sep 14 01:48:17.075934 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:17.075898    2581 scope.go:117] "RemoveContainer" containerID="cae769fd0b5c243f84fefad95a0a51c412ddc7adc472856f313593b7528e65b6"
Sep 14 01:48:17.076282 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:48:17.076264    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cae769fd0b5c243f84fefad95a0a51c412ddc7adc472856f313593b7528e65b6\": container with ID starting with cae769fd0b5c243f84fefad95a0a51c412ddc7adc472856f313593b7528e65b6 not found: ID does not exist" containerID="cae769fd0b5c243f84fefad95a0a51c412ddc7adc472856f313593b7528e65b6"
Sep 14 01:48:17.076338 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:17.076292    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cae769fd0b5c243f84fefad95a0a51c412ddc7adc472856f313593b7528e65b6"} err="failed to get container status \"cae769fd0b5c243f84fefad95a0a51c412ddc7adc472856f313593b7528e65b6\": rpc error: code = NotFound desc = could not find container \"cae769fd0b5c243f84fefad95a0a51c412ddc7adc472856f313593b7528e65b6\": container with ID starting with cae769fd0b5c243f84fefad95a0a51c412ddc7adc472856f313593b7528e65b6 not found: ID does not exist"
Sep 14 01:48:17.082504 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:17.082480    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"]
Sep 14 01:48:17.085823 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:17.085798    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-bc2e37-7cb54f794b-v84r7"]
Sep 14 01:48:19.054836 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:19.054795    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9909d893-a3d9-45ce-bd57-3a1ed91a344d" path="/var/lib/kubelet/pods/9909d893-a3d9-45ce-bd57-3a1ed91a344d/volumes"
Sep 14 01:48:21.685619 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:21.685578    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"]
Sep 14 01:48:23.211243 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:23.211201    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:23.211631 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:23.211262    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:23.223977 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:23.223946    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:24.119020 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:24.118992    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:48:31.426468 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:31.426433    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"]
Sep 14 01:48:31.427051 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:31.426686    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff" podUID="292cfd39-635f-4dfc-9e4c-176b77ad5ea1" containerName="maas-api" containerID="cri-o://44c5e2068be7e4d822944db6907e25c3199be8612bdc54c9b515b2e23fbf7c2e" gracePeriod=30
Sep 14 01:48:31.687782 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:31.687716    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:31.695005 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:31.694981    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-metrics-certs\") pod \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") "
Sep 14 01:48:31.695126 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:31.695089    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-maas-api-tls\") pod \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") "
Sep 14 01:48:31.695126 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:31.695108    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-openshift-service-ca\") pod \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") "
Sep 14 01:48:31.695241 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:31.695156    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ks525\" (UniqueName: \"kubernetes.io/projected/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-kube-api-access-ks525\") pod \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\" (UID: \"292cfd39-635f-4dfc-9e4c-176b77ad5ea1\") "
Sep 14 01:48:31.695498 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:31.695469    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "292cfd39-635f-4dfc-9e4c-176b77ad5ea1" (UID: "292cfd39-635f-4dfc-9e4c-176b77ad5ea1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:48:31.697254 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:31.697229    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-kube-api-access-ks525" (OuterVolumeSpecName: "kube-api-access-ks525") pod "292cfd39-635f-4dfc-9e4c-176b77ad5ea1" (UID: "292cfd39-635f-4dfc-9e4c-176b77ad5ea1"). InnerVolumeSpecName "kube-api-access-ks525". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:48:31.697342 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:31.697232    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "292cfd39-635f-4dfc-9e4c-176b77ad5ea1" (UID: "292cfd39-635f-4dfc-9e4c-176b77ad5ea1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:48:31.697342 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:31.697259    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "292cfd39-635f-4dfc-9e4c-176b77ad5ea1" (UID: "292cfd39-635f-4dfc-9e4c-176b77ad5ea1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:48:31.796192 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:31.796157    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:48:31.796192 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:31.796186    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:48:31.796192 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:31.796197    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ks525\" (UniqueName: \"kubernetes.io/projected/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-kube-api-access-ks525\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:48:31.796437 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:31.796206    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/292cfd39-635f-4dfc-9e4c-176b77ad5ea1-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:48:32.155307 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:32.155275    2581 generic.go:356] "Generic (PLEG): container finished" podID="292cfd39-635f-4dfc-9e4c-176b77ad5ea1" containerID="44c5e2068be7e4d822944db6907e25c3199be8612bdc54c9b515b2e23fbf7c2e" exitCode=0
Sep 14 01:48:32.155531 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:32.155325    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"
Sep 14 01:48:32.155531 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:32.155375    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff" event={"ID":"292cfd39-635f-4dfc-9e4c-176b77ad5ea1","Type":"ContainerDied","Data":"44c5e2068be7e4d822944db6907e25c3199be8612bdc54c9b515b2e23fbf7c2e"}
Sep 14 01:48:32.155531 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:32.155437    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff" event={"ID":"292cfd39-635f-4dfc-9e4c-176b77ad5ea1","Type":"ContainerDied","Data":"c80a549abe8f938e4422b82815b059f6e91bc73b0562440636b03183f8c5f508"}
Sep 14 01:48:32.155531 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:32.155468    2581 scope.go:117] "RemoveContainer" containerID="44c5e2068be7e4d822944db6907e25c3199be8612bdc54c9b515b2e23fbf7c2e"
Sep 14 01:48:32.165803 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:32.165782    2581 scope.go:117] "RemoveContainer" containerID="44c5e2068be7e4d822944db6907e25c3199be8612bdc54c9b515b2e23fbf7c2e"
Sep 14 01:48:32.166131 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:48:32.166110    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44c5e2068be7e4d822944db6907e25c3199be8612bdc54c9b515b2e23fbf7c2e\": container with ID starting with 44c5e2068be7e4d822944db6907e25c3199be8612bdc54c9b515b2e23fbf7c2e not found: ID does not exist" containerID="44c5e2068be7e4d822944db6907e25c3199be8612bdc54c9b515b2e23fbf7c2e"
Sep 14 01:48:32.166217 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:32.166141    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44c5e2068be7e4d822944db6907e25c3199be8612bdc54c9b515b2e23fbf7c2e"} err="failed to get container status \"44c5e2068be7e4d822944db6907e25c3199be8612bdc54c9b515b2e23fbf7c2e\": rpc error: code = NotFound desc = could not find container \"44c5e2068be7e4d822944db6907e25c3199be8612bdc54c9b515b2e23fbf7c2e\": container with ID starting with 44c5e2068be7e4d822944db6907e25c3199be8612bdc54c9b515b2e23fbf7c2e not found: ID does not exist"
Sep 14 01:48:32.179850 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:32.179821    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"]
Sep 14 01:48:32.182149 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:32.182130    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b351ffdb-6cd4bcc747-hkdff"]
Sep 14 01:48:33.054350 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:33.054316    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="292cfd39-635f-4dfc-9e4c-176b77ad5ea1" path="/var/lib/kubelet/pods/292cfd39-635f-4dfc-9e4c-176b77ad5ea1/volumes"
Sep 14 01:48:36.705325 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:36.705261    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t" podUID="628b4a0b-9431-4434-8bd4-25c602fccda0" containerName="main" containerID="cri-o://87d82c230be82a6236a490df93bd3fbd19d8c2e12eb3989eb5e1b05b3809de0b" gracePeriod=45
Sep 14 01:48:36.854313 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:36.854277    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:48:36.969024 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:36.969000    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:48:37.040136 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.040105    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjbpw\" (UniqueName: \"kubernetes.io/projected/628b4a0b-9431-4434-8bd4-25c602fccda0-kube-api-access-tjbpw\") pod \"628b4a0b-9431-4434-8bd4-25c602fccda0\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") "
Sep 14 01:48:37.040136 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.040152    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628b4a0b-9431-4434-8bd4-25c602fccda0-tls-certs\") pod \"628b4a0b-9431-4434-8bd4-25c602fccda0\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") "
Sep 14 01:48:37.040409 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.040203    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-dshm\") pod \"628b4a0b-9431-4434-8bd4-25c602fccda0\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") "
Sep 14 01:48:37.040409 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.040229    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-tmp-dir\") pod \"628b4a0b-9431-4434-8bd4-25c602fccda0\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") "
Sep 14 01:48:37.040409 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.040254    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-model-cache\") pod \"628b4a0b-9431-4434-8bd4-25c602fccda0\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") "
Sep 14 01:48:37.040409 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.040324    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-home\") pod \"628b4a0b-9431-4434-8bd4-25c602fccda0\" (UID: \"628b4a0b-9431-4434-8bd4-25c602fccda0\") "
Sep 14 01:48:37.040612 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.040524    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "628b4a0b-9431-4434-8bd4-25c602fccda0" (UID: "628b4a0b-9431-4434-8bd4-25c602fccda0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:48:37.040612 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.040571    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-model-cache" (OuterVolumeSpecName: "model-cache") pod "628b4a0b-9431-4434-8bd4-25c602fccda0" (UID: "628b4a0b-9431-4434-8bd4-25c602fccda0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:48:37.040760 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.040741    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-home" (OuterVolumeSpecName: "home") pod "628b4a0b-9431-4434-8bd4-25c602fccda0" (UID: "628b4a0b-9431-4434-8bd4-25c602fccda0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:48:37.040820 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.040765    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:48:37.040820 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.040786    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:48:37.042664 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.042636    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-dshm" (OuterVolumeSpecName: "dshm") pod "628b4a0b-9431-4434-8bd4-25c602fccda0" (UID: "628b4a0b-9431-4434-8bd4-25c602fccda0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:48:37.042863 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.042841    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/628b4a0b-9431-4434-8bd4-25c602fccda0-kube-api-access-tjbpw" (OuterVolumeSpecName: "kube-api-access-tjbpw") pod "628b4a0b-9431-4434-8bd4-25c602fccda0" (UID: "628b4a0b-9431-4434-8bd4-25c602fccda0"). InnerVolumeSpecName "kube-api-access-tjbpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:48:37.042944 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.042862    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/628b4a0b-9431-4434-8bd4-25c602fccda0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "628b4a0b-9431-4434-8bd4-25c602fccda0" (UID: "628b4a0b-9431-4434-8bd4-25c602fccda0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:48:37.142289 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.142247    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjbpw\" (UniqueName: \"kubernetes.io/projected/628b4a0b-9431-4434-8bd4-25c602fccda0-kube-api-access-tjbpw\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:48:37.142289 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.142281    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628b4a0b-9431-4434-8bd4-25c602fccda0-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:48:37.142289 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.142293    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:48:37.142537 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.142303    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/628b4a0b-9431-4434-8bd4-25c602fccda0-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:48:37.186730 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.186695    2581 generic.go:356] "Generic (PLEG): container finished" podID="628b4a0b-9431-4434-8bd4-25c602fccda0" containerID="87d82c230be82a6236a490df93bd3fbd19d8c2e12eb3989eb5e1b05b3809de0b" exitCode=0
Sep 14 01:48:37.186948 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.186737    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"
Sep 14 01:48:37.186948 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.186790    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t" event={"ID":"628b4a0b-9431-4434-8bd4-25c602fccda0","Type":"ContainerDied","Data":"87d82c230be82a6236a490df93bd3fbd19d8c2e12eb3989eb5e1b05b3809de0b"}
Sep 14 01:48:37.186948 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.186849    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t" event={"ID":"628b4a0b-9431-4434-8bd4-25c602fccda0","Type":"ContainerDied","Data":"36d2cad79b4f8a2dd4f730b6bfbb13a6e6d3caf317c73458246dedfa2fb8f3ab"}
Sep 14 01:48:37.186948 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.186873    2581 scope.go:117] "RemoveContainer" containerID="87d82c230be82a6236a490df93bd3fbd19d8c2e12eb3989eb5e1b05b3809de0b"
Sep 14 01:48:37.196357 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.196331    2581 scope.go:117] "RemoveContainer" containerID="87d82c230be82a6236a490df93bd3fbd19d8c2e12eb3989eb5e1b05b3809de0b"
Sep 14 01:48:37.196728 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:48:37.196698    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87d82c230be82a6236a490df93bd3fbd19d8c2e12eb3989eb5e1b05b3809de0b\": container with ID starting with 87d82c230be82a6236a490df93bd3fbd19d8c2e12eb3989eb5e1b05b3809de0b not found: ID does not exist" containerID="87d82c230be82a6236a490df93bd3fbd19d8c2e12eb3989eb5e1b05b3809de0b"
Sep 14 01:48:37.196842 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.196740    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87d82c230be82a6236a490df93bd3fbd19d8c2e12eb3989eb5e1b05b3809de0b"} err="failed to get container status \"87d82c230be82a6236a490df93bd3fbd19d8c2e12eb3989eb5e1b05b3809de0b\": rpc error: code = NotFound desc = could not find container \"87d82c230be82a6236a490df93bd3fbd19d8c2e12eb3989eb5e1b05b3809de0b\": container with ID starting with 87d82c230be82a6236a490df93bd3fbd19d8c2e12eb3989eb5e1b05b3809de0b not found: ID does not exist"
Sep 14 01:48:37.201822 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.201787    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"]
Sep 14 01:48:37.204425 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:37.204403    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-bc2e37/autores-model-bc2e37-kserve-5dbccdbff5-qcp9t"]
Sep 14 01:48:39.055004 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:48:39.054970    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="628b4a0b-9431-4434-8bd4-25c602fccda0" path="/var/lib/kubelet/pods/628b4a0b-9431-4434-8bd4-25c602fccda0/volumes"
Sep 14 01:49:05.114751 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.114708    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"]
Sep 14 01:49:05.115385 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.115343    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9909d893-a3d9-45ce-bd57-3a1ed91a344d" containerName="maas-api"
Sep 14 01:49:05.115385 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.115364    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9909d893-a3d9-45ce-bd57-3a1ed91a344d" containerName="maas-api"
Sep 14 01:49:05.115385 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.115385    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="628b4a0b-9431-4434-8bd4-25c602fccda0" containerName="main"
Sep 14 01:49:05.115554 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.115393    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="628b4a0b-9431-4434-8bd4-25c602fccda0" containerName="main"
Sep 14 01:49:05.115554 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.115406    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="292cfd39-635f-4dfc-9e4c-176b77ad5ea1" containerName="maas-api"
Sep 14 01:49:05.115554 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.115416    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="292cfd39-635f-4dfc-9e4c-176b77ad5ea1" containerName="maas-api"
Sep 14 01:49:05.115554 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.115518    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="628b4a0b-9431-4434-8bd4-25c602fccda0" containerName="main"
Sep 14 01:49:05.115554 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.115533    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9909d893-a3d9-45ce-bd57-3a1ed91a344d" containerName="maas-api"
Sep 14 01:49:05.115554 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.115541    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="292cfd39-635f-4dfc-9e4c-176b77ad5ea1" containerName="maas-api"
Sep 14 01:49:05.120861 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.120833    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:05.122690 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.122664    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-7aab97-serving-cert\""
Sep 14 01:49:05.128993 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.128960    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"]
Sep 14 01:49:05.226248 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.226213    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c34a8445-7a80-4634-b839-dd50e42edb4b-metrics-certs\") pod \"maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:05.226466 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.226257    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n29kg\" (UniqueName: \"kubernetes.io/projected/c34a8445-7a80-4634-b839-dd50e42edb4b-kube-api-access-n29kg\") pod \"maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:05.226466 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.226405    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c34a8445-7a80-4634-b839-dd50e42edb4b-openshift-service-ca\") pod \"maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:05.226466 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.226454    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c34a8445-7a80-4634-b839-dd50e42edb4b-maas-api-tls\") pod \"maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:05.327865 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.327825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n29kg\" (UniqueName: \"kubernetes.io/projected/c34a8445-7a80-4634-b839-dd50e42edb4b-kube-api-access-n29kg\") pod \"maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:05.328098 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.327952    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c34a8445-7a80-4634-b839-dd50e42edb4b-openshift-service-ca\") pod \"maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:05.328098 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.327993    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c34a8445-7a80-4634-b839-dd50e42edb4b-maas-api-tls\") pod \"maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:05.328098 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.328072    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c34a8445-7a80-4634-b839-dd50e42edb4b-metrics-certs\") pod \"maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:05.328690 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.328663    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c34a8445-7a80-4634-b839-dd50e42edb4b-openshift-service-ca\") pod \"maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:05.330644 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.330622    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c34a8445-7a80-4634-b839-dd50e42edb4b-maas-api-tls\") pod \"maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:05.330743 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.330670    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c34a8445-7a80-4634-b839-dd50e42edb4b-metrics-certs\") pod \"maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:05.334755 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.334731    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n29kg\" (UniqueName: \"kubernetes.io/projected/c34a8445-7a80-4634-b839-dd50e42edb4b-kube-api-access-n29kg\") pod \"maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:05.435744 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.435650    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:05.711149 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:05.711117    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"]
Sep 14 01:49:05.712740 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:49:05.712714    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc34a8445_7a80_4634_b839_dd50e42edb4b.slice/crio-00a4abe09074caabe7eac74011823ff99bfc174f03c51c724f7ed235fc5c59a6 WatchSource:0}: Error finding container 00a4abe09074caabe7eac74011823ff99bfc174f03c51c724f7ed235fc5c59a6: Status 404 returned error can't find the container with id 00a4abe09074caabe7eac74011823ff99bfc174f03c51c724f7ed235fc5c59a6
Sep 14 01:49:06.350539 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:06.350505    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99" event={"ID":"c34a8445-7a80-4634-b839-dd50e42edb4b","Type":"ContainerStarted","Data":"e8805e5e693044d12e33ef4ac43bc7ac5627fd09a1d51048c802073a08ea07f9"}
Sep 14 01:49:06.350964 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:06.350564    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99" event={"ID":"c34a8445-7a80-4634-b839-dd50e42edb4b","Type":"ContainerStarted","Data":"00a4abe09074caabe7eac74011823ff99bfc174f03c51c724f7ed235fc5c59a6"}
Sep 14 01:49:06.350964 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:06.350613    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:06.366702 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:06.366601    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99" podStartSLOduration=0.965301087 podStartE2EDuration="1.366582211s" totalImagesPullingTime="401.281124ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:49:06.363975169 +0000 UTC m=+1105.844425989" watchObservedRunningTime="2026-09-14 01:49:06.366582211 +0000 UTC m=+1105.847033031"
Sep 14 01:49:08.396234 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.396197    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"]
Sep 14 01:49:08.399576 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.399560    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.401396 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.401374    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-99c616\"/\"premium-w0-99c616-kserve-self-signed-certs\""
Sep 14 01:49:08.406280 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.406255    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"]
Sep 14 01:49:08.557172 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.557130    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-home\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.557359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.557178    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-dshm\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.557359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.557258    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-model-cache\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.557359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.557339    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-594fk\" (UniqueName: \"kubernetes.io/projected/163fadc1-20b0-4343-9f44-fc45d3730dec-kube-api-access-594fk\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.557471 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.557370    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-tmp-dir\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.557471 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.557409    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163fadc1-20b0-4343-9f44-fc45d3730dec-tls-certs\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.658887 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.658793    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163fadc1-20b0-4343-9f44-fc45d3730dec-tls-certs\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.658887 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.658874    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-home\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.659166 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.658908    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-dshm\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.659166 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.658952    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-model-cache\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.659166 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.658992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-594fk\" (UniqueName: \"kubernetes.io/projected/163fadc1-20b0-4343-9f44-fc45d3730dec-kube-api-access-594fk\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.659166 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.659018    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-tmp-dir\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.659406 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.659373    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-home\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.659543 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.659416    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-model-cache\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.659543 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.659468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-tmp-dir\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.661324 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.661298    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-dshm\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.661620 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.661602    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163fadc1-20b0-4343-9f44-fc45d3730dec-tls-certs\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.666003 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.665982    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-594fk\" (UniqueName: \"kubernetes.io/projected/163fadc1-20b0-4343-9f44-fc45d3730dec-kube-api-access-594fk\") pod \"premium-w0-99c616-kserve-748b9cbb4d-qsz68\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") " pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.710378 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.710336    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:08.986769 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:08.986744    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"]
Sep 14 01:49:08.989012 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:49:08.988977    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod163fadc1_20b0_4343_9f44_fc45d3730dec.slice/crio-17bff756da8843fd0f4fc57b0f24cc4d92596c8bb19115c850e6b556732f5d2e WatchSource:0}: Error finding container 17bff756da8843fd0f4fc57b0f24cc4d92596c8bb19115c850e6b556732f5d2e: Status 404 returned error can't find the container with id 17bff756da8843fd0f4fc57b0f24cc4d92596c8bb19115c850e6b556732f5d2e
Sep 14 01:49:09.368869 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:09.368774    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68" event={"ID":"163fadc1-20b0-4343-9f44-fc45d3730dec","Type":"ContainerStarted","Data":"805c36780cab8504ba6f24165d208234ff79552f9127d7ae7b50bd3b6a094c4c"}
Sep 14 01:49:09.368869 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:09.368823    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68" event={"ID":"163fadc1-20b0-4343-9f44-fc45d3730dec","Type":"ContainerStarted","Data":"17bff756da8843fd0f4fc57b0f24cc4d92596c8bb19115c850e6b556732f5d2e"}
Sep 14 01:49:09.385728 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:09.385675    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68" podStartSLOduration=1.385659924 podStartE2EDuration="1.385659924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:49:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:49:09.382777224 +0000 UTC m=+1108.863228045" watchObservedRunningTime="2026-09-14 01:49:09.385659924 +0000 UTC m=+1108.866110761"
Sep 14 01:49:12.362151 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:12.362122    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:49:14.892966 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:14.892865    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:49:18.710959 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:18.710898    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:18.710959 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:18.710966    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:18.723342 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:18.723309    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:19.437792 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:19.437759    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:49:33.610338 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.610293    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"]
Sep 14 01:49:33.615226 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.615203    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.617346 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.617325    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-8e5935\"/\"premium-w4-8e5935-kserve-self-signed-certs\""
Sep 14 01:49:33.623219 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.623190    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"]
Sep 14 01:49:33.713588 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.713548    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-tmp-dir\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.713754 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.713601    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnjr9\" (UniqueName: \"kubernetes.io/projected/161d40d2-885a-44f6-8541-b84681091614-kube-api-access-cnjr9\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.713754 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.713730    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-home\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.713839 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.713769    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/161d40d2-885a-44f6-8541-b84681091614-tls-certs\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.713839 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.713804    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-model-cache\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.713839 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.713837    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-dshm\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.815367 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.815328    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-dshm\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.815580 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.815379    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-tmp-dir\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.815580 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.815410    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnjr9\" (UniqueName: \"kubernetes.io/projected/161d40d2-885a-44f6-8541-b84681091614-kube-api-access-cnjr9\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.815580 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.815470    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-home\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.815753 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.815584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/161d40d2-885a-44f6-8541-b84681091614-tls-certs\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.815753 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.815619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-model-cache\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.815883 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.815858    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-tmp-dir\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.815954 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.815890    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-home\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.816025 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.816005    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-model-cache\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.817687 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.817667    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-dshm\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.818095 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.818079    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/161d40d2-885a-44f6-8541-b84681091614-tls-certs\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.824234 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.824207    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnjr9\" (UniqueName: \"kubernetes.io/projected/161d40d2-885a-44f6-8541-b84681091614-kube-api-access-cnjr9\") pod \"premium-w4-8e5935-kserve-758bccfd47-9dcp5\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") " pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:33.926144 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:33.926037    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:34.199758 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:34.199727    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"]
Sep 14 01:49:34.201733 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:49:34.201700    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod161d40d2_885a_44f6_8541_b84681091614.slice/crio-02c325e9ff44536c79a2b0d1b06cd8c4d10005be175ec6e45f5a7d2fb195f3a5 WatchSource:0}: Error finding container 02c325e9ff44536c79a2b0d1b06cd8c4d10005be175ec6e45f5a7d2fb195f3a5: Status 404 returned error can't find the container with id 02c325e9ff44536c79a2b0d1b06cd8c4d10005be175ec6e45f5a7d2fb195f3a5
Sep 14 01:49:34.515094 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:34.515058    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5" event={"ID":"161d40d2-885a-44f6-8541-b84681091614","Type":"ContainerStarted","Data":"7b44c23b4816d3c2d090a4f912e1513f570b086c8489d483d43fad78d4bb8e80"}
Sep 14 01:49:34.515245 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:34.515103    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5" event={"ID":"161d40d2-885a-44f6-8541-b84681091614","Type":"ContainerStarted","Data":"02c325e9ff44536c79a2b0d1b06cd8c4d10005be175ec6e45f5a7d2fb195f3a5"}
Sep 14 01:49:34.538070 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:34.538017    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5" podStartSLOduration=1.53799896 podStartE2EDuration="1.53799896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:49:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:49:34.535724875 +0000 UTC m=+1134.016175697" watchObservedRunningTime="2026-09-14 01:49:34.53799896 +0000 UTC m=+1134.018449788"
Sep 14 01:49:40.764851 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:40.764809    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:49:43.926236 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:43.926178    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:43.926679 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:43.926257    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:43.939375 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:43.939347    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:44.584760 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:44.584732    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 01:49:58.357634 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:49:58.357593    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:50:14.409164 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:14.408964    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"]
Sep 14 01:50:14.413690 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:14.413665    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:14.417458 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:14.417408    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-287e139e-serving-cert\""
Sep 14 01:50:14.420846 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:14.420815    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"]
Sep 14 01:50:14.514641 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:14.514603    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxdtp\" (UniqueName: \"kubernetes.io/projected/4f009fd7-0100-4452-a607-5bfe147a44df-kube-api-access-fxdtp\") pod \"maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:14.514831 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:14.514716    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f009fd7-0100-4452-a607-5bfe147a44df-maas-api-tls\") pod \"maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:14.514831 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:14.514773    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f009fd7-0100-4452-a607-5bfe147a44df-openshift-service-ca\") pod \"maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:14.514907 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:14.514859    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f009fd7-0100-4452-a607-5bfe147a44df-metrics-certs\") pod \"maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:14.616416 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:14.616379    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f009fd7-0100-4452-a607-5bfe147a44df-metrics-certs\") pod \"maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:14.616583 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:14.616460    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxdtp\" (UniqueName: \"kubernetes.io/projected/4f009fd7-0100-4452-a607-5bfe147a44df-kube-api-access-fxdtp\") pod \"maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:14.616583 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:14.616520    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f009fd7-0100-4452-a607-5bfe147a44df-maas-api-tls\") pod \"maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:14.616692 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:14.616579    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f009fd7-0100-4452-a607-5bfe147a44df-openshift-service-ca\") pod \"maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:14.616748 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:50:14.616695    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-serving-cert: secret "maas-api-e2e-mt-287e139e-serving-cert" not found
Sep 14 01:50:14.616820 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:50:14.616792    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f009fd7-0100-4452-a607-5bfe147a44df-maas-api-tls podName:4f009fd7-0100-4452-a607-5bfe147a44df nodeName:}" failed. No retries permitted until 2026-09-14 01:50:15.116767604 +0000 UTC m=+1174.597218420 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4f009fd7-0100-4452-a607-5bfe147a44df-maas-api-tls") pod "maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp" (UID: "4f009fd7-0100-4452-a607-5bfe147a44df") : secret "maas-api-e2e-mt-287e139e-serving-cert" not found
Sep 14 01:50:14.617379 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:14.617360    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f009fd7-0100-4452-a607-5bfe147a44df-openshift-service-ca\") pod \"maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:14.619170 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:14.619150    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f009fd7-0100-4452-a607-5bfe147a44df-metrics-certs\") pod \"maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:14.624689 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:14.624659    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxdtp\" (UniqueName: \"kubernetes.io/projected/4f009fd7-0100-4452-a607-5bfe147a44df-kube-api-access-fxdtp\") pod \"maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:15.130970 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:15.130882    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f009fd7-0100-4452-a607-5bfe147a44df-maas-api-tls\") pod \"maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:15.133419 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:15.133393    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f009fd7-0100-4452-a607-5bfe147a44df-maas-api-tls\") pod \"maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:15.328496 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:15.328461    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:15.600315 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:15.600290    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"]
Sep 14 01:50:15.601789 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:50:15.601764    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f009fd7_0100_4452_a607_5bfe147a44df.slice/crio-54279105347c6cf775e43dd002fc34bb5fe858e433a02cd123e1e5adfe08713e WatchSource:0}: Error finding container 54279105347c6cf775e43dd002fc34bb5fe858e433a02cd123e1e5adfe08713e: Status 404 returned error can't find the container with id 54279105347c6cf775e43dd002fc34bb5fe858e433a02cd123e1e5adfe08713e
Sep 14 01:50:15.757494 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:15.757459    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp" event={"ID":"4f009fd7-0100-4452-a607-5bfe147a44df","Type":"ContainerStarted","Data":"54279105347c6cf775e43dd002fc34bb5fe858e433a02cd123e1e5adfe08713e"}
Sep 14 01:50:16.765447 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:16.765401    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp" event={"ID":"4f009fd7-0100-4452-a607-5bfe147a44df","Type":"ContainerStarted","Data":"3563beb5b85a6388dc4a34041bab963d658d0b5be33dc9de3861e38e1fc69d54"}
Sep 14 01:50:16.765833 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:16.765478    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:16.781929 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:16.781870    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp" podStartSLOduration=2.450487827 podStartE2EDuration="2.781855242s" totalImagesPullingTime="331.367415ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:50:16.779477368 +0000 UTC m=+1176.259928188" watchObservedRunningTime="2026-09-14 01:50:16.781855242 +0000 UTC m=+1176.262306063"
Sep 14 01:50:22.777571 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:22.777539    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:27.470471 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:27.470431    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:50:33.222752 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.222719    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"]
Sep 14 01:50:33.226790 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.226764    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.228948 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.228902    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-321811\"/\"kube-root-ca.crt\""
Sep 14 01:50:33.229641 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.229602    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-321811\"/\"default-dockercfg-5w7bw\""
Sep 14 01:50:33.229765 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.229640    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-321811\"/\"test-model-321811-kserve-self-signed-certs\""
Sep 14 01:50:33.229765 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.229640    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-321811\"/\"openshift-service-ca.crt\""
Sep 14 01:50:33.234170 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.234148    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"]
Sep 14 01:50:33.298663 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.298621    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-model-cache\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.298851 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.298672    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64vxn\" (UniqueName: \"kubernetes.io/projected/7502f0c1-553d-4d21-ab16-dbb00afa7f62-kube-api-access-64vxn\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.298851 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.298755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-tmp-dir\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.298851 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.298796    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-home\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.298851 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.298816    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-dshm\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.299027 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.298936    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7502f0c1-553d-4d21-ab16-dbb00afa7f62-tls-certs\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.400290 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.400250    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64vxn\" (UniqueName: \"kubernetes.io/projected/7502f0c1-553d-4d21-ab16-dbb00afa7f62-kube-api-access-64vxn\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.400504 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.400323    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-tmp-dir\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.400504 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.400374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-home\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.400504 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.400400    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-dshm\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.400504 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.400488    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7502f0c1-553d-4d21-ab16-dbb00afa7f62-tls-certs\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.400721 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.400527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-model-cache\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.400848 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.400816    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-home\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.400980 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.400830    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-tmp-dir\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.400980 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.400861    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-model-cache\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.402740 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.402717    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-dshm\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.403190 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.403168    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7502f0c1-553d-4d21-ab16-dbb00afa7f62-tls-certs\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.407483 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.407463    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64vxn\" (UniqueName: \"kubernetes.io/projected/7502f0c1-553d-4d21-ab16-dbb00afa7f62-kube-api-access-64vxn\") pod \"test-model-321811-kserve-754db89857-c6ggk\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") " pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.538716 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.538634    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:33.817421 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.817386    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"]
Sep 14 01:50:33.819316 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:50:33.819285    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7502f0c1_553d_4d21_ab16_dbb00afa7f62.slice/crio-de691f661337abc3a15d4117ca4ffff1f090c4c51897ef17dfd8d564b2427ace WatchSource:0}: Error finding container de691f661337abc3a15d4117ca4ffff1f090c4c51897ef17dfd8d564b2427ace: Status 404 returned error can't find the container with id de691f661337abc3a15d4117ca4ffff1f090c4c51897ef17dfd8d564b2427ace
Sep 14 01:50:33.868787 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:33.868757    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk" event={"ID":"7502f0c1-553d-4d21-ab16-dbb00afa7f62","Type":"ContainerStarted","Data":"de691f661337abc3a15d4117ca4ffff1f090c4c51897ef17dfd8d564b2427ace"}
Sep 14 01:50:34.876723 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:34.876690    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk" event={"ID":"7502f0c1-553d-4d21-ab16-dbb00afa7f62","Type":"ContainerStarted","Data":"0cac9a534ef987d339f4034d5d50d62fcc837f8854f2e320882b3614789c72cd"}
Sep 14 01:50:34.894488 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:34.894436    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk" podStartSLOduration=1.8944197699999998 podStartE2EDuration="1.89441977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:50:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:50:34.891618495 +0000 UTC m=+1194.372069316" watchObservedRunningTime="2026-09-14 01:50:34.89441977 +0000 UTC m=+1194.374870591"
Sep 14 01:50:41.102406 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:41.102378    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 01:50:41.104334 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:41.104309    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 01:50:43.539022 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:43.538973    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:43.541445 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:43.539052    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:43.552487 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:43.552457    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:43.943293 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:43.943202    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:50:57.858103 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:57.858061    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:50:58.412722 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:58.412685    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"]
Sep 14 01:50:58.413029 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:58.412995    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp" podUID="4f009fd7-0100-4452-a607-5bfe147a44df" containerName="maas-api" containerID="cri-o://3563beb5b85a6388dc4a34041bab963d658d0b5be33dc9de3861e38e1fc69d54" gracePeriod=30
Sep 14 01:50:58.672630 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:58.672563    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:58.767553 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:58.767512    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f009fd7-0100-4452-a607-5bfe147a44df-maas-api-tls\") pod \"4f009fd7-0100-4452-a607-5bfe147a44df\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") "
Sep 14 01:50:58.767721 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:58.767596    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f009fd7-0100-4452-a607-5bfe147a44df-metrics-certs\") pod \"4f009fd7-0100-4452-a607-5bfe147a44df\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") "
Sep 14 01:50:58.767721 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:58.767660    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxdtp\" (UniqueName: \"kubernetes.io/projected/4f009fd7-0100-4452-a607-5bfe147a44df-kube-api-access-fxdtp\") pod \"4f009fd7-0100-4452-a607-5bfe147a44df\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") "
Sep 14 01:50:58.767721 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:58.767699    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f009fd7-0100-4452-a607-5bfe147a44df-openshift-service-ca\") pod \"4f009fd7-0100-4452-a607-5bfe147a44df\" (UID: \"4f009fd7-0100-4452-a607-5bfe147a44df\") "
Sep 14 01:50:58.768189 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:58.768155    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f009fd7-0100-4452-a607-5bfe147a44df-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4f009fd7-0100-4452-a607-5bfe147a44df" (UID: "4f009fd7-0100-4452-a607-5bfe147a44df"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:50:58.770421 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:58.770381    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f009fd7-0100-4452-a607-5bfe147a44df-kube-api-access-fxdtp" (OuterVolumeSpecName: "kube-api-access-fxdtp") pod "4f009fd7-0100-4452-a607-5bfe147a44df" (UID: "4f009fd7-0100-4452-a607-5bfe147a44df"). InnerVolumeSpecName "kube-api-access-fxdtp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:50:58.770421 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:58.770405    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f009fd7-0100-4452-a607-5bfe147a44df-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4f009fd7-0100-4452-a607-5bfe147a44df" (UID: "4f009fd7-0100-4452-a607-5bfe147a44df"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:50:58.770617 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:58.770498    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f009fd7-0100-4452-a607-5bfe147a44df-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4f009fd7-0100-4452-a607-5bfe147a44df" (UID: "4f009fd7-0100-4452-a607-5bfe147a44df"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:50:58.869523 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:58.869473    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxdtp\" (UniqueName: \"kubernetes.io/projected/4f009fd7-0100-4452-a607-5bfe147a44df-kube-api-access-fxdtp\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:50:58.869523 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:58.869512    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f009fd7-0100-4452-a607-5bfe147a44df-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:50:58.869523 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:58.869523    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f009fd7-0100-4452-a607-5bfe147a44df-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:50:58.869523 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:58.869532    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f009fd7-0100-4452-a607-5bfe147a44df-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:50:59.032523 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:59.032489    2581 generic.go:356] "Generic (PLEG): container finished" podID="4f009fd7-0100-4452-a607-5bfe147a44df" containerID="3563beb5b85a6388dc4a34041bab963d658d0b5be33dc9de3861e38e1fc69d54" exitCode=0
Sep 14 01:50:59.032703 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:59.032542    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"
Sep 14 01:50:59.032703 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:59.032576    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp" event={"ID":"4f009fd7-0100-4452-a607-5bfe147a44df","Type":"ContainerDied","Data":"3563beb5b85a6388dc4a34041bab963d658d0b5be33dc9de3861e38e1fc69d54"}
Sep 14 01:50:59.032703 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:59.032638    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp" event={"ID":"4f009fd7-0100-4452-a607-5bfe147a44df","Type":"ContainerDied","Data":"54279105347c6cf775e43dd002fc34bb5fe858e433a02cd123e1e5adfe08713e"}
Sep 14 01:50:59.032703 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:59.032667    2581 scope.go:117] "RemoveContainer" containerID="3563beb5b85a6388dc4a34041bab963d658d0b5be33dc9de3861e38e1fc69d54"
Sep 14 01:50:59.042713 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:59.042694    2581 scope.go:117] "RemoveContainer" containerID="3563beb5b85a6388dc4a34041bab963d658d0b5be33dc9de3861e38e1fc69d54"
Sep 14 01:50:59.043065 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:50:59.043043    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3563beb5b85a6388dc4a34041bab963d658d0b5be33dc9de3861e38e1fc69d54\": container with ID starting with 3563beb5b85a6388dc4a34041bab963d658d0b5be33dc9de3861e38e1fc69d54 not found: ID does not exist" containerID="3563beb5b85a6388dc4a34041bab963d658d0b5be33dc9de3861e38e1fc69d54"
Sep 14 01:50:59.043129 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:59.043074    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3563beb5b85a6388dc4a34041bab963d658d0b5be33dc9de3861e38e1fc69d54"} err="failed to get container status \"3563beb5b85a6388dc4a34041bab963d658d0b5be33dc9de3861e38e1fc69d54\": rpc error: code = NotFound desc = could not find container \"3563beb5b85a6388dc4a34041bab963d658d0b5be33dc9de3861e38e1fc69d54\": container with ID starting with 3563beb5b85a6388dc4a34041bab963d658d0b5be33dc9de3861e38e1fc69d54 not found: ID does not exist"
Sep 14 01:50:59.055260 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:59.055233    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"]
Sep 14 01:50:59.056202 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:50:59.056180    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-287e139e-868cc4dbb8-h49rp"]
Sep 14 01:51:01.054138 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:01.054096    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f009fd7-0100-4452-a607-5bfe147a44df" path="/var/lib/kubelet/pods/4f009fd7-0100-4452-a607-5bfe147a44df/volumes"
Sep 14 01:51:18.262382 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:18.262345    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:51:24.909593 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:24.909554    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"]
Sep 14 01:51:24.910050 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:24.909983    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f009fd7-0100-4452-a607-5bfe147a44df" containerName="maas-api"
Sep 14 01:51:24.910050 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:24.909996    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f009fd7-0100-4452-a607-5bfe147a44df" containerName="maas-api"
Sep 14 01:51:24.910129 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:24.910101    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f009fd7-0100-4452-a607-5bfe147a44df" containerName="maas-api"
Sep 14 01:51:24.913784 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:24.913760    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:24.915796 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:24.915773    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-38e192-serving-cert\""
Sep 14 01:51:24.924746 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:24.924724    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"]
Sep 14 01:51:25.043059 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:25.043017    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvqm9\" (UniqueName: \"kubernetes.io/projected/5f1c351f-d602-413a-8ac3-584bc7e21146-kube-api-access-gvqm9\") pod \"maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:25.043259 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:25.043066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f1c351f-d602-413a-8ac3-584bc7e21146-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:25.043259 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:25.043121    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1c351f-d602-413a-8ac3-584bc7e21146-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:25.043259 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:25.043237    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1c351f-d602-413a-8ac3-584bc7e21146-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:25.144161 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:25.144121    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvqm9\" (UniqueName: \"kubernetes.io/projected/5f1c351f-d602-413a-8ac3-584bc7e21146-kube-api-access-gvqm9\") pod \"maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:25.144377 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:25.144196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f1c351f-d602-413a-8ac3-584bc7e21146-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:25.144377 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:25.144248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1c351f-d602-413a-8ac3-584bc7e21146-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:25.144377 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:25.144328    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1c351f-d602-413a-8ac3-584bc7e21146-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:25.144593 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:51:25.144441    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-serving-cert: secret "maas-api-e2e-shared-a-w2-38e192-serving-cert" not found
Sep 14 01:51:25.144593 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:51:25.144527    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1c351f-d602-413a-8ac3-584bc7e21146-maas-api-tls podName:5f1c351f-d602-413a-8ac3-584bc7e21146 nodeName:}" failed. No retries permitted until 2026-09-14 01:51:25.644503844 +0000 UTC m=+1245.124954662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5f1c351f-d602-413a-8ac3-584bc7e21146-maas-api-tls") pod "maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z" (UID: "5f1c351f-d602-413a-8ac3-584bc7e21146") : secret "maas-api-e2e-shared-a-w2-38e192-serving-cert" not found
Sep 14 01:51:25.145176 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:25.145148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f1c351f-d602-413a-8ac3-584bc7e21146-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:25.146941 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:25.146901    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1c351f-d602-413a-8ac3-584bc7e21146-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:25.152137 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:25.152115    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvqm9\" (UniqueName: \"kubernetes.io/projected/5f1c351f-d602-413a-8ac3-584bc7e21146-kube-api-access-gvqm9\") pod \"maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:25.650174 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:25.650137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1c351f-d602-413a-8ac3-584bc7e21146-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:25.652575 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:25.652545    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1c351f-d602-413a-8ac3-584bc7e21146-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:25.827287 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:25.827252    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:26.097566 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:26.097530    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"]
Sep 14 01:51:26.099718 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:51:26.099692    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f1c351f_d602_413a_8ac3_584bc7e21146.slice/crio-1b38eea6ef2b89a4dcc335846a491bc0d43b3c8aed02279f7bea14b6a39ca29c WatchSource:0}: Error finding container 1b38eea6ef2b89a4dcc335846a491bc0d43b3c8aed02279f7bea14b6a39ca29c: Status 404 returned error can't find the container with id 1b38eea6ef2b89a4dcc335846a491bc0d43b3c8aed02279f7bea14b6a39ca29c
Sep 14 01:51:26.101274 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:26.101253    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 01:51:26.189966 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:26.189929    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z" event={"ID":"5f1c351f-d602-413a-8ac3-584bc7e21146","Type":"ContainerStarted","Data":"1b38eea6ef2b89a4dcc335846a491bc0d43b3c8aed02279f7bea14b6a39ca29c"}
Sep 14 01:51:27.201036 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:27.200996    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z" event={"ID":"5f1c351f-d602-413a-8ac3-584bc7e21146","Type":"ContainerStarted","Data":"e75c67410999a6d76a3cfbfde979f102fe7f9e9bd5595114d4ff5418b9e3ddea"}
Sep 14 01:51:27.201495 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:27.201189    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:27.218724 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:27.218674    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z" podStartSLOduration=2.831259659 podStartE2EDuration="3.21865972s" totalImagesPullingTime="387.400061ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:51:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:51:27.216843321 +0000 UTC m=+1246.697294143" watchObservedRunningTime="2026-09-14 01:51:27.21865972 +0000 UTC m=+1246.699110541"
Sep 14 01:51:33.220081 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:33.220007    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:51:36.265190 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:36.265149    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:51:40.816437 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:40.816402    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"]
Sep 14 01:51:40.816930 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:40.816636    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99" podUID="c34a8445-7a80-4634-b839-dd50e42edb4b" containerName="maas-api" containerID="cri-o://e8805e5e693044d12e33ef4ac43bc7ac5627fd09a1d51048c802073a08ea07f9" gracePeriod=30
Sep 14 01:51:41.081564 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.081498    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:51:41.207875 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.207843    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c34a8445-7a80-4634-b839-dd50e42edb4b-openshift-service-ca\") pod \"c34a8445-7a80-4634-b839-dd50e42edb4b\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") "
Sep 14 01:51:41.208085 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.207894    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n29kg\" (UniqueName: \"kubernetes.io/projected/c34a8445-7a80-4634-b839-dd50e42edb4b-kube-api-access-n29kg\") pod \"c34a8445-7a80-4634-b839-dd50e42edb4b\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") "
Sep 14 01:51:41.208085 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.208002    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c34a8445-7a80-4634-b839-dd50e42edb4b-maas-api-tls\") pod \"c34a8445-7a80-4634-b839-dd50e42edb4b\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") "
Sep 14 01:51:41.208085 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.208046    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c34a8445-7a80-4634-b839-dd50e42edb4b-metrics-certs\") pod \"c34a8445-7a80-4634-b839-dd50e42edb4b\" (UID: \"c34a8445-7a80-4634-b839-dd50e42edb4b\") "
Sep 14 01:51:41.208251 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.208220    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c34a8445-7a80-4634-b839-dd50e42edb4b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c34a8445-7a80-4634-b839-dd50e42edb4b" (UID: "c34a8445-7a80-4634-b839-dd50e42edb4b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:51:41.208364 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.208341    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c34a8445-7a80-4634-b839-dd50e42edb4b-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:51:41.210161 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.210133    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c34a8445-7a80-4634-b839-dd50e42edb4b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c34a8445-7a80-4634-b839-dd50e42edb4b" (UID: "c34a8445-7a80-4634-b839-dd50e42edb4b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:51:41.210256 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.210216    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c34a8445-7a80-4634-b839-dd50e42edb4b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c34a8445-7a80-4634-b839-dd50e42edb4b" (UID: "c34a8445-7a80-4634-b839-dd50e42edb4b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:51:41.210256 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.210236    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c34a8445-7a80-4634-b839-dd50e42edb4b-kube-api-access-n29kg" (OuterVolumeSpecName: "kube-api-access-n29kg") pod "c34a8445-7a80-4634-b839-dd50e42edb4b" (UID: "c34a8445-7a80-4634-b839-dd50e42edb4b"). InnerVolumeSpecName "kube-api-access-n29kg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:51:41.300211 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.300171    2581 generic.go:356] "Generic (PLEG): container finished" podID="c34a8445-7a80-4634-b839-dd50e42edb4b" containerID="e8805e5e693044d12e33ef4ac43bc7ac5627fd09a1d51048c802073a08ea07f9" exitCode=0
Sep 14 01:51:41.300378 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.300220    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"
Sep 14 01:51:41.300378 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.300236    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99" event={"ID":"c34a8445-7a80-4634-b839-dd50e42edb4b","Type":"ContainerDied","Data":"e8805e5e693044d12e33ef4ac43bc7ac5627fd09a1d51048c802073a08ea07f9"}
Sep 14 01:51:41.300378 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.300291    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99" event={"ID":"c34a8445-7a80-4634-b839-dd50e42edb4b","Type":"ContainerDied","Data":"00a4abe09074caabe7eac74011823ff99bfc174f03c51c724f7ed235fc5c59a6"}
Sep 14 01:51:41.300378 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.300323    2581 scope.go:117] "RemoveContainer" containerID="e8805e5e693044d12e33ef4ac43bc7ac5627fd09a1d51048c802073a08ea07f9"
Sep 14 01:51:41.308922 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.308887    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n29kg\" (UniqueName: \"kubernetes.io/projected/c34a8445-7a80-4634-b839-dd50e42edb4b-kube-api-access-n29kg\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:51:41.308922 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.308925    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c34a8445-7a80-4634-b839-dd50e42edb4b-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:51:41.309074 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.308936    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c34a8445-7a80-4634-b839-dd50e42edb4b-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:51:41.310318 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.310305    2581 scope.go:117] "RemoveContainer" containerID="e8805e5e693044d12e33ef4ac43bc7ac5627fd09a1d51048c802073a08ea07f9"
Sep 14 01:51:41.310624 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:51:41.310604    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8805e5e693044d12e33ef4ac43bc7ac5627fd09a1d51048c802073a08ea07f9\": container with ID starting with e8805e5e693044d12e33ef4ac43bc7ac5627fd09a1d51048c802073a08ea07f9 not found: ID does not exist" containerID="e8805e5e693044d12e33ef4ac43bc7ac5627fd09a1d51048c802073a08ea07f9"
Sep 14 01:51:41.310670 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.310635    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8805e5e693044d12e33ef4ac43bc7ac5627fd09a1d51048c802073a08ea07f9"} err="failed to get container status \"e8805e5e693044d12e33ef4ac43bc7ac5627fd09a1d51048c802073a08ea07f9\": rpc error: code = NotFound desc = could not find container \"e8805e5e693044d12e33ef4ac43bc7ac5627fd09a1d51048c802073a08ea07f9\": container with ID starting with e8805e5e693044d12e33ef4ac43bc7ac5627fd09a1d51048c802073a08ea07f9 not found: ID does not exist"
Sep 14 01:51:41.320869 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.320840    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"]
Sep 14 01:51:41.324722 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:41.324698    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-7aab97-85d9bdbb56-5tc99"]
Sep 14 01:51:43.055164 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:43.055122    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c34a8445-7a80-4634-b839-dd50e42edb4b" path="/var/lib/kubelet/pods/c34a8445-7a80-4634-b839-dd50e42edb4b/volumes"
Sep 14 01:51:54.527630 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.527586    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"]
Sep 14 01:51:54.528233 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.528036    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c34a8445-7a80-4634-b839-dd50e42edb4b" containerName="maas-api"
Sep 14 01:51:54.528233 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.528055    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c34a8445-7a80-4634-b839-dd50e42edb4b" containerName="maas-api"
Sep 14 01:51:54.528233 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.528210    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c34a8445-7a80-4634-b839-dd50e42edb4b" containerName="maas-api"
Sep 14 01:51:54.537988 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.537961    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:54.540005 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.539975    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-7ff93e-serving-cert\""
Sep 14 01:51:54.547409 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.546785    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"]
Sep 14 01:51:54.639247 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.639212    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:54.639433 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.639275    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgtvd\" (UniqueName: \"kubernetes.io/projected/e9e6352f-059f-4a13-b214-52799a9fbc55-kube-api-access-mgtvd\") pod \"maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:54.639433 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.639335    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9e6352f-059f-4a13-b214-52799a9fbc55-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:54.639508 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.639462    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:54.740621 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.740573    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:54.740808 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.740637    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:54.740808 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.740687    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgtvd\" (UniqueName: \"kubernetes.io/projected/e9e6352f-059f-4a13-b214-52799a9fbc55-kube-api-access-mgtvd\") pod \"maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:54.740808 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.740752    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9e6352f-059f-4a13-b214-52799a9fbc55-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:54.740808 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:51:54.740754    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-serving-cert: secret "maas-api-e2e-shared-b-w2-7ff93e-serving-cert" not found
Sep 14 01:51:54.741078 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:51:54.740848    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-maas-api-tls podName:e9e6352f-059f-4a13-b214-52799a9fbc55 nodeName:}" failed. No retries permitted until 2026-09-14 01:51:55.240826647 +0000 UTC m=+1274.721277449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-maas-api-tls") pod "maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf" (UID: "e9e6352f-059f-4a13-b214-52799a9fbc55") : secret "maas-api-e2e-shared-b-w2-7ff93e-serving-cert" not found
Sep 14 01:51:54.741503 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.741479    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9e6352f-059f-4a13-b214-52799a9fbc55-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:54.743477 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.743454    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:54.751237 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:54.751209    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgtvd\" (UniqueName: \"kubernetes.io/projected/e9e6352f-059f-4a13-b214-52799a9fbc55-kube-api-access-mgtvd\") pod \"maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:55.245067 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:55.245031    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:55.245249 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:51:55.245187    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-serving-cert: secret "maas-api-e2e-shared-b-w2-7ff93e-serving-cert" not found
Sep 14 01:51:55.245293 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:51:55.245275    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-maas-api-tls podName:e9e6352f-059f-4a13-b214-52799a9fbc55 nodeName:}" failed. No retries permitted until 2026-09-14 01:51:56.245258732 +0000 UTC m=+1275.725709551 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-maas-api-tls") pod "maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf" (UID: "e9e6352f-059f-4a13-b214-52799a9fbc55") : secret "maas-api-e2e-shared-b-w2-7ff93e-serving-cert" not found
Sep 14 01:51:56.255295 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:56.255261    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:56.257675 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:56.257655    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:56.351405 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:56.351369    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:56.623725 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:56.623683    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"]
Sep 14 01:51:56.625395 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:51:56.625360    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9e6352f_059f_4a13_b214_52799a9fbc55.slice/crio-71d3bdae8f1ca07f426fa23ce68b1c2398501a76d3ee857e4989058a4ce469da WatchSource:0}: Error finding container 71d3bdae8f1ca07f426fa23ce68b1c2398501a76d3ee857e4989058a4ce469da: Status 404 returned error can't find the container with id 71d3bdae8f1ca07f426fa23ce68b1c2398501a76d3ee857e4989058a4ce469da
Sep 14 01:51:57.403034 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:57.402943    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf" event={"ID":"e9e6352f-059f-4a13-b214-52799a9fbc55","Type":"ContainerStarted","Data":"9d3785d577a8380dc2e8b8b0697cee0f36a907db36b3b864cc1aaf3e98a4f150"}
Sep 14 01:51:57.403034 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:57.402996    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf" event={"ID":"e9e6352f-059f-4a13-b214-52799a9fbc55","Type":"ContainerStarted","Data":"71d3bdae8f1ca07f426fa23ce68b1c2398501a76d3ee857e4989058a4ce469da"}
Sep 14 01:51:57.403460 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:57.403085    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:51:57.419236 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:51:57.419189    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf" podStartSLOduration=3.031606695 podStartE2EDuration="3.419172319s" totalImagesPullingTime="387.565624ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:51:57.416479485 +0000 UTC m=+1276.896930304" watchObservedRunningTime="2026-09-14 01:51:57.419172319 +0000 UTC m=+1276.899623145"
Sep 14 01:52:03.414540 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:03.414501    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:52:04.462069 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:04.462035    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:52:21.381841 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:21.381812    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7443-937a92"]
Sep 14 01:52:21.385419 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:21.385399    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-937a92"
Sep 14 01:52:21.387630 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:21.387611    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-74qx8\""
Sep 14 01:52:21.393045 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:21.393020    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7443-937a92"]
Sep 14 01:52:21.512743 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:21.512707    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtzh7\" (UniqueName: \"kubernetes.io/projected/033baec3-4e16-473d-bd85-fdf3c4496f9d-kube-api-access-qtzh7\") pod \"test-curl-7443-937a92\" (UID: \"033baec3-4e16-473d-bd85-fdf3c4496f9d\") " pod="opendatahub/test-curl-7443-937a92"
Sep 14 01:52:21.614136 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:21.614097    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtzh7\" (UniqueName: \"kubernetes.io/projected/033baec3-4e16-473d-bd85-fdf3c4496f9d-kube-api-access-qtzh7\") pod \"test-curl-7443-937a92\" (UID: \"033baec3-4e16-473d-bd85-fdf3c4496f9d\") " pod="opendatahub/test-curl-7443-937a92"
Sep 14 01:52:21.623564 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:21.623534    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtzh7\" (UniqueName: \"kubernetes.io/projected/033baec3-4e16-473d-bd85-fdf3c4496f9d-kube-api-access-qtzh7\") pod \"test-curl-7443-937a92\" (UID: \"033baec3-4e16-473d-bd85-fdf3c4496f9d\") " pod="opendatahub/test-curl-7443-937a92"
Sep 14 01:52:21.696854 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:21.696766    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-937a92"
Sep 14 01:52:21.966855 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:21.966828    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7443-937a92"]
Sep 14 01:52:21.968232 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:52:21.968204    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod033baec3_4e16_473d_bd85_fdf3c4496f9d.slice/crio-4e37e1395678227b4d1251981486050272fea36b1ed9528a53b354f53dfface5 WatchSource:0}: Error finding container 4e37e1395678227b4d1251981486050272fea36b1ed9528a53b354f53dfface5: Status 404 returned error can't find the container with id 4e37e1395678227b4d1251981486050272fea36b1ed9528a53b354f53dfface5
Sep 14 01:52:22.565673 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:22.565641    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7443-937a92" event={"ID":"033baec3-4e16-473d-bd85-fdf3c4496f9d","Type":"ContainerStarted","Data":"fc2c59ea200343549d9d85b2a85344bdd4e51b818547c947f966f1d3016d6300"}
Sep 14 01:52:22.565673 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:22.565684    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7443-937a92" event={"ID":"033baec3-4e16-473d-bd85-fdf3c4496f9d","Type":"ContainerStarted","Data":"4e37e1395678227b4d1251981486050272fea36b1ed9528a53b354f53dfface5"}
Sep 14 01:52:22.578190 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:22.578138    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7443-937a92" podStartSLOduration=1.5781201820000001 podStartE2EDuration="1.578120182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:52:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:52:22.577524972 +0000 UTC m=+1302.057975793" watchObservedRunningTime="2026-09-14 01:52:22.578120182 +0000 UTC m=+1302.058571006"
Sep 14 01:52:23.123383 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:23.123349    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7443-937a92"]
Sep 14 01:52:23.127425 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:23.127396    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7443-937a92"]
Sep 14 01:52:23.233372 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:23.233339    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7443-c4ed83"]
Sep 14 01:52:23.233813 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:23.233798    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="033baec3-4e16-473d-bd85-fdf3c4496f9d" containerName="test-curl-7443-937a92"
Sep 14 01:52:23.233869 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:23.233816    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="033baec3-4e16-473d-bd85-fdf3c4496f9d" containerName="test-curl-7443-937a92"
Sep 14 01:52:23.233905 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:23.233892    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="033baec3-4e16-473d-bd85-fdf3c4496f9d" containerName="test-curl-7443-937a92"
Sep 14 01:52:23.237194 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:23.237177    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-c4ed83"
Sep 14 01:52:23.244451 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:23.244419    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7443-c4ed83"]
Sep 14 01:52:23.331742 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:23.331707    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hn7n\" (UniqueName: \"kubernetes.io/projected/e942081f-0370-49f9-96b1-858be7676104-kube-api-access-8hn7n\") pod \"test-curl-7443-c4ed83\" (UID: \"e942081f-0370-49f9-96b1-858be7676104\") " pod="opendatahub/test-curl-7443-c4ed83"
Sep 14 01:52:23.432737 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:23.432697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hn7n\" (UniqueName: \"kubernetes.io/projected/e942081f-0370-49f9-96b1-858be7676104-kube-api-access-8hn7n\") pod \"test-curl-7443-c4ed83\" (UID: \"e942081f-0370-49f9-96b1-858be7676104\") " pod="opendatahub/test-curl-7443-c4ed83"
Sep 14 01:52:23.441509 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:23.441486    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hn7n\" (UniqueName: \"kubernetes.io/projected/e942081f-0370-49f9-96b1-858be7676104-kube-api-access-8hn7n\") pod \"test-curl-7443-c4ed83\" (UID: \"e942081f-0370-49f9-96b1-858be7676104\") " pod="opendatahub/test-curl-7443-c4ed83"
Sep 14 01:52:23.547748 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:23.547648    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-c4ed83"
Sep 14 01:52:23.573299 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:23.573275    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7443-937a92" container="test-curl-7443-937a92"
Sep 14 01:52:23.829447 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:23.829421    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7443-c4ed83"]
Sep 14 01:52:23.831293 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:52:23.831265    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode942081f_0370_49f9_96b1_858be7676104.slice/crio-fd9ea53dd00380591c823c536b717cab7ca3e34779bdbe0dfcecc41f004f246e WatchSource:0}: Error finding container fd9ea53dd00380591c823c536b717cab7ca3e34779bdbe0dfcecc41f004f246e: Status 404 returned error can't find the container with id fd9ea53dd00380591c823c536b717cab7ca3e34779bdbe0dfcecc41f004f246e
Sep 14 01:52:24.581069 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:24.581036    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7443-c4ed83" event={"ID":"e942081f-0370-49f9-96b1-858be7676104","Type":"ContainerStarted","Data":"4f9d563795895d0af66ede6fdcc643407606fad7139d916b903d428fa52f8042"}
Sep 14 01:52:24.581549 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:24.581081    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7443-c4ed83" event={"ID":"e942081f-0370-49f9-96b1-858be7676104","Type":"ContainerStarted","Data":"fd9ea53dd00380591c823c536b717cab7ca3e34779bdbe0dfcecc41f004f246e"}
Sep 14 01:52:24.581549 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:24.581132    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7443-937a92" podUID="033baec3-4e16-473d-bd85-fdf3c4496f9d" containerName="test-curl-7443-937a92" containerID="cri-o://fc2c59ea200343549d9d85b2a85344bdd4e51b818547c947f966f1d3016d6300" gracePeriod=2
Sep 14 01:52:24.592937 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:24.592862    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7443-c4ed83" podStartSLOduration=1.5928442 podStartE2EDuration="1.5928442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:52:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:52:24.592628932 +0000 UTC m=+1304.073079753" watchObservedRunningTime="2026-09-14 01:52:24.5928442 +0000 UTC m=+1304.073295023"
Sep 14 01:52:24.594082 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:24.594051    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7443-937a92\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="033baec3-4e16-473d-bd85-fdf3c4496f9d" pod="opendatahub/test-curl-7443-937a92"
Sep 14 01:52:24.595403 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:24.595385    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7443-937a92\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="033baec3-4e16-473d-bd85-fdf3c4496f9d" pod="opendatahub/test-curl-7443-937a92"
Sep 14 01:52:25.150046 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:25.150014    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7443-c4ed83"]
Sep 14 01:52:25.154405 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:25.154379    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7443-c4ed83"]
Sep 14 01:52:25.258815 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:25.258778    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7443-a6d4c3"]
Sep 14 01:52:25.259310 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:25.259291    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e942081f-0370-49f9-96b1-858be7676104" containerName="test-curl-7443-c4ed83"
Sep 14 01:52:25.259413 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:25.259313    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e942081f-0370-49f9-96b1-858be7676104" containerName="test-curl-7443-c4ed83"
Sep 14 01:52:25.259465 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:25.259447    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e942081f-0370-49f9-96b1-858be7676104" containerName="test-curl-7443-c4ed83"
Sep 14 01:52:25.262776 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:25.262701    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-a6d4c3"
Sep 14 01:52:25.266208 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:25.266186    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7443-a6d4c3"]
Sep 14 01:52:25.349809 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:25.349768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlpgr\" (UniqueName: \"kubernetes.io/projected/a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc-kube-api-access-xlpgr\") pod \"test-curl-7443-a6d4c3\" (UID: \"a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc\") " pod="opendatahub/test-curl-7443-a6d4c3"
Sep 14 01:52:25.450724 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:25.450670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlpgr\" (UniqueName: \"kubernetes.io/projected/a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc-kube-api-access-xlpgr\") pod \"test-curl-7443-a6d4c3\" (UID: \"a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc\") " pod="opendatahub/test-curl-7443-a6d4c3"
Sep 14 01:52:25.458405 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:25.458378    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlpgr\" (UniqueName: \"kubernetes.io/projected/a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc-kube-api-access-xlpgr\") pod \"test-curl-7443-a6d4c3\" (UID: \"a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc\") " pod="opendatahub/test-curl-7443-a6d4c3"
Sep 14 01:52:25.574502 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:25.574409    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-a6d4c3"
Sep 14 01:52:25.588402 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:25.588371    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7443-c4ed83" container="test-curl-7443-c4ed83"
Sep 14 01:52:25.850395 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:25.850362    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7443-a6d4c3"]
Sep 14 01:52:25.851955 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:52:25.851905    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda189cf5e_be53_4ceb_b5ea_91bfaf9e12fc.slice/crio-3001dccd8189be3e4e3808793f5ddebab35dfeecd9eccd5687dfd90b2a2a479f WatchSource:0}: Error finding container 3001dccd8189be3e4e3808793f5ddebab35dfeecd9eccd5687dfd90b2a2a479f: Status 404 returned error can't find the container with id 3001dccd8189be3e4e3808793f5ddebab35dfeecd9eccd5687dfd90b2a2a479f
Sep 14 01:52:25.964486 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:25.964452    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:52:26.596707 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:26.596670    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7443-a6d4c3" event={"ID":"a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc","Type":"ContainerStarted","Data":"29eccdf158891ddcdf9aa307bde9a64ec8af5b8271a6faa57c83dae10dbe4a25"}
Sep 14 01:52:26.597032 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:26.596722    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7443-a6d4c3" event={"ID":"a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc","Type":"ContainerStarted","Data":"3001dccd8189be3e4e3808793f5ddebab35dfeecd9eccd5687dfd90b2a2a479f"}
Sep 14 01:52:26.597032 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:26.596938    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7443-c4ed83" podUID="e942081f-0370-49f9-96b1-858be7676104" containerName="test-curl-7443-c4ed83" containerID="cri-o://4f9d563795895d0af66ede6fdcc643407606fad7139d916b903d428fa52f8042" gracePeriod=2
Sep 14 01:52:26.613411 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:26.613360    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7443-a6d4c3" podStartSLOduration=1.613345485 podStartE2EDuration="1.613345485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:52:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:52:26.612879182 +0000 UTC m=+1306.093330003" watchObservedRunningTime="2026-09-14 01:52:26.613345485 +0000 UTC m=+1306.093796306"
Sep 14 01:52:26.614490 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:26.614460    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7443-c4ed83\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="e942081f-0370-49f9-96b1-858be7676104" pod="opendatahub/test-curl-7443-c4ed83"
Sep 14 01:52:26.786209 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:26.786182    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-937a92"
Sep 14 01:52:26.787998 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:26.787971    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7443-c4ed83\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="e942081f-0370-49f9-96b1-858be7676104" pod="opendatahub/test-curl-7443-c4ed83"
Sep 14 01:52:26.868762 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:26.868675    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtzh7\" (UniqueName: \"kubernetes.io/projected/033baec3-4e16-473d-bd85-fdf3c4496f9d-kube-api-access-qtzh7\") pod \"033baec3-4e16-473d-bd85-fdf3c4496f9d\" (UID: \"033baec3-4e16-473d-bd85-fdf3c4496f9d\") "
Sep 14 01:52:26.870851 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:26.870820    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/033baec3-4e16-473d-bd85-fdf3c4496f9d-kube-api-access-qtzh7" (OuterVolumeSpecName: "kube-api-access-qtzh7") pod "033baec3-4e16-473d-bd85-fdf3c4496f9d" (UID: "033baec3-4e16-473d-bd85-fdf3c4496f9d"). InnerVolumeSpecName "kube-api-access-qtzh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:52:26.970629 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:26.970588    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtzh7\" (UniqueName: \"kubernetes.io/projected/033baec3-4e16-473d-bd85-fdf3c4496f9d-kube-api-access-qtzh7\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:52:27.055123 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.055086    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="033baec3-4e16-473d-bd85-fdf3c4496f9d" path="/var/lib/kubelet/pods/033baec3-4e16-473d-bd85-fdf3c4496f9d/volumes"
Sep 14 01:52:27.173452 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.173355    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7443-a6d4c3"]
Sep 14 01:52:27.176087 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.176060    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7443-a6d4c3"]
Sep 14 01:52:27.277222 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.277186    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7443-90142c"]
Sep 14 01:52:27.277610 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.277598    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc" containerName="test-curl-7443-a6d4c3"
Sep 14 01:52:27.277654 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.277612    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc" containerName="test-curl-7443-a6d4c3"
Sep 14 01:52:27.277695 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.277683    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc" containerName="test-curl-7443-a6d4c3"
Sep 14 01:52:27.281042 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.281020    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-90142c"
Sep 14 01:52:27.286359 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.286333    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7443-90142c"]
Sep 14 01:52:27.375295 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.375256    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql6rf\" (UniqueName: \"kubernetes.io/projected/52049aad-5081-4d84-aeb5-c1db17c79f6d-kube-api-access-ql6rf\") pod \"test-curl-7443-90142c\" (UID: \"52049aad-5081-4d84-aeb5-c1db17c79f6d\") " pod="opendatahub/test-curl-7443-90142c"
Sep 14 01:52:27.476237 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.476195    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql6rf\" (UniqueName: \"kubernetes.io/projected/52049aad-5081-4d84-aeb5-c1db17c79f6d-kube-api-access-ql6rf\") pod \"test-curl-7443-90142c\" (UID: \"52049aad-5081-4d84-aeb5-c1db17c79f6d\") " pod="opendatahub/test-curl-7443-90142c"
Sep 14 01:52:27.483371 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.483343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql6rf\" (UniqueName: \"kubernetes.io/projected/52049aad-5081-4d84-aeb5-c1db17c79f6d-kube-api-access-ql6rf\") pod \"test-curl-7443-90142c\" (UID: \"52049aad-5081-4d84-aeb5-c1db17c79f6d\") " pod="opendatahub/test-curl-7443-90142c"
Sep 14 01:52:27.591382 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.591349    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-90142c"
Sep 14 01:52:27.603470 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.603435    2581 generic.go:356] "Generic (PLEG): container finished" podID="033baec3-4e16-473d-bd85-fdf3c4496f9d" containerID="fc2c59ea200343549d9d85b2a85344bdd4e51b818547c947f966f1d3016d6300" exitCode=137
Sep 14 01:52:27.603858 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.603480    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-937a92"
Sep 14 01:52:27.603858 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.603519    2581 scope.go:117] "RemoveContainer" containerID="fc2c59ea200343549d9d85b2a85344bdd4e51b818547c947f966f1d3016d6300"
Sep 14 01:52:27.603858 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:27.603845    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7443-a6d4c3" container="test-curl-7443-a6d4c3"
Sep 14 01:52:27.626038 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.626019    2581 scope.go:117] "RemoveContainer" containerID="fc2c59ea200343549d9d85b2a85344bdd4e51b818547c947f966f1d3016d6300"
Sep 14 01:52:27.626427 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:27.626404    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc2c59ea200343549d9d85b2a85344bdd4e51b818547c947f966f1d3016d6300\": container with ID starting with fc2c59ea200343549d9d85b2a85344bdd4e51b818547c947f966f1d3016d6300 not found: ID does not exist" containerID="fc2c59ea200343549d9d85b2a85344bdd4e51b818547c947f966f1d3016d6300"
Sep 14 01:52:27.626525 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.626437    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc2c59ea200343549d9d85b2a85344bdd4e51b818547c947f966f1d3016d6300"} err="failed to get container status \"fc2c59ea200343549d9d85b2a85344bdd4e51b818547c947f966f1d3016d6300\": rpc error: code = NotFound desc = could not find container \"fc2c59ea200343549d9d85b2a85344bdd4e51b818547c947f966f1d3016d6300\": container with ID starting with fc2c59ea200343549d9d85b2a85344bdd4e51b818547c947f966f1d3016d6300 not found: ID does not exist"
Sep 14 01:52:27.863621 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:27.863594    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7443-90142c"]
Sep 14 01:52:27.865004 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:52:27.864975    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52049aad_5081_4d84_aeb5_c1db17c79f6d.slice/crio-4e9e4534e2ac73899fe75be4b9d61467b4138e59e3cd71d5fb0b3c8244fb2a9b WatchSource:0}: Error finding container 4e9e4534e2ac73899fe75be4b9d61467b4138e59e3cd71d5fb0b3c8244fb2a9b: Status 404 returned error can't find the container with id 4e9e4534e2ac73899fe75be4b9d61467b4138e59e3cd71d5fb0b3c8244fb2a9b
Sep 14 01:52:28.612351 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:28.612307    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7443-90142c" event={"ID":"52049aad-5081-4d84-aeb5-c1db17c79f6d","Type":"ContainerStarted","Data":"7827b9dca520827460d9051cefc6664cb8cff8c0dfe629fa22013ddc0be218ed"}
Sep 14 01:52:28.612671 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:28.612357    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7443-90142c" event={"ID":"52049aad-5081-4d84-aeb5-c1db17c79f6d","Type":"ContainerStarted","Data":"4e9e4534e2ac73899fe75be4b9d61467b4138e59e3cd71d5fb0b3c8244fb2a9b"}
Sep 14 01:52:28.613632 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:28.613600    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7443-a6d4c3" podUID="a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc" containerName="test-curl-7443-a6d4c3" containerID="cri-o://29eccdf158891ddcdf9aa307bde9a64ec8af5b8271a6faa57c83dae10dbe4a25" gracePeriod=2
Sep 14 01:52:28.626512 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:28.626474    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7443-a6d4c3\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc" pod="opendatahub/test-curl-7443-a6d4c3"
Sep 14 01:52:28.627848 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:28.627808    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7443-90142c" podStartSLOduration=1.627795727 podStartE2EDuration="1.627795727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:52:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:52:28.624941871 +0000 UTC m=+1308.105392691" watchObservedRunningTime="2026-09-14 01:52:28.627795727 +0000 UTC m=+1308.108246547"
Sep 14 01:52:28.796879 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:28.796851    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-c4ed83"
Sep 14 01:52:28.798365 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:28.798337    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7443-a6d4c3\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc" pod="opendatahub/test-curl-7443-a6d4c3"
Sep 14 01:52:28.890488 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:28.890400    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hn7n\" (UniqueName: \"kubernetes.io/projected/e942081f-0370-49f9-96b1-858be7676104-kube-api-access-8hn7n\") pod \"e942081f-0370-49f9-96b1-858be7676104\" (UID: \"e942081f-0370-49f9-96b1-858be7676104\") "
Sep 14 01:52:28.892770 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:28.892686    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e942081f-0370-49f9-96b1-858be7676104-kube-api-access-8hn7n" (OuterVolumeSpecName: "kube-api-access-8hn7n") pod "e942081f-0370-49f9-96b1-858be7676104" (UID: "e942081f-0370-49f9-96b1-858be7676104"). InnerVolumeSpecName "kube-api-access-8hn7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:52:28.991267 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:28.991231    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hn7n\" (UniqueName: \"kubernetes.io/projected/e942081f-0370-49f9-96b1-858be7676104-kube-api-access-8hn7n\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:52:29.054773 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:29.054737    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e942081f-0370-49f9-96b1-858be7676104" path="/var/lib/kubelet/pods/e942081f-0370-49f9-96b1-858be7676104/volumes"
Sep 14 01:52:29.187220 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:29.187130    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7443-90142c"]
Sep 14 01:52:29.192844 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:29.192816    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7443-90142c"]
Sep 14 01:52:29.628831 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:29.628793    2581 generic.go:356] "Generic (PLEG): container finished" podID="e942081f-0370-49f9-96b1-858be7676104" containerID="4f9d563795895d0af66ede6fdcc643407606fad7139d916b903d428fa52f8042" exitCode=137
Sep 14 01:52:29.629304 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:29.628842    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-c4ed83"
Sep 14 01:52:29.629304 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:29.628890    2581 scope.go:117] "RemoveContainer" containerID="4f9d563795895d0af66ede6fdcc643407606fad7139d916b903d428fa52f8042"
Sep 14 01:52:29.629304 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:29.629125    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7443-90142c" container="test-curl-7443-90142c"
Sep 14 01:52:29.638141 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:29.638125    2581 scope.go:117] "RemoveContainer" containerID="4f9d563795895d0af66ede6fdcc643407606fad7139d916b903d428fa52f8042"
Sep 14 01:52:29.638431 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:29.638409    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f9d563795895d0af66ede6fdcc643407606fad7139d916b903d428fa52f8042\": container with ID starting with 4f9d563795895d0af66ede6fdcc643407606fad7139d916b903d428fa52f8042 not found: ID does not exist" containerID="4f9d563795895d0af66ede6fdcc643407606fad7139d916b903d428fa52f8042"
Sep 14 01:52:29.638505 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:29.638440    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f9d563795895d0af66ede6fdcc643407606fad7139d916b903d428fa52f8042"} err="failed to get container status \"4f9d563795895d0af66ede6fdcc643407606fad7139d916b903d428fa52f8042\": rpc error: code = NotFound desc = could not find container \"4f9d563795895d0af66ede6fdcc643407606fad7139d916b903d428fa52f8042\": container with ID starting with 4f9d563795895d0af66ede6fdcc643407606fad7139d916b903d428fa52f8042 not found: ID does not exist"
Sep 14 01:52:30.636125 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:30.636091    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7443-90142c" podUID="52049aad-5081-4d84-aeb5-c1db17c79f6d" containerName="test-curl-7443-90142c" containerID="cri-o://7827b9dca520827460d9051cefc6664cb8cff8c0dfe629fa22013ddc0be218ed" gracePeriod=2
Sep 14 01:52:30.638338 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:30.638313    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7443-90142c\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="52049aad-5081-4d84-aeb5-c1db17c79f6d" pod="opendatahub/test-curl-7443-90142c"
Sep 14 01:52:30.797158 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:30.797136    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-a6d4c3"
Sep 14 01:52:30.798721 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:30.798691    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7443-90142c\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="52049aad-5081-4d84-aeb5-c1db17c79f6d" pod="opendatahub/test-curl-7443-90142c"
Sep 14 01:52:30.912165 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:30.912071    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlpgr\" (UniqueName: \"kubernetes.io/projected/a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc-kube-api-access-xlpgr\") pod \"a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc\" (UID: \"a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc\") "
Sep 14 01:52:30.914190 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:30.914166    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc-kube-api-access-xlpgr" (OuterVolumeSpecName: "kube-api-access-xlpgr") pod "a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc" (UID: "a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc"). InnerVolumeSpecName "kube-api-access-xlpgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:52:31.013435 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:31.013399    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlpgr\" (UniqueName: \"kubernetes.io/projected/a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc-kube-api-access-xlpgr\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:52:31.052974 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:31.052942    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7443-90142c\" is forbidden: User \"system:node:ip-10-0-132-11.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-11.ec2.internal' and this object" podUID="52049aad-5081-4d84-aeb5-c1db17c79f6d" pod="opendatahub/test-curl-7443-90142c"
Sep 14 01:52:31.055397 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:31.055376    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc" path="/var/lib/kubelet/pods/a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc/volumes"
Sep 14 01:52:31.643276 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:31.643244    2581 generic.go:356] "Generic (PLEG): container finished" podID="a189cf5e-be53-4ceb-b5ea-91bfaf9e12fc" containerID="29eccdf158891ddcdf9aa307bde9a64ec8af5b8271a6faa57c83dae10dbe4a25" exitCode=137
Sep 14 01:52:31.643702 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:31.643283    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-a6d4c3"
Sep 14 01:52:31.643702 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:31.643338    2581 scope.go:117] "RemoveContainer" containerID="29eccdf158891ddcdf9aa307bde9a64ec8af5b8271a6faa57c83dae10dbe4a25"
Sep 14 01:52:31.652791 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:31.652768    2581 scope.go:117] "RemoveContainer" containerID="29eccdf158891ddcdf9aa307bde9a64ec8af5b8271a6faa57c83dae10dbe4a25"
Sep 14 01:52:31.653130 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:31.653109    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29eccdf158891ddcdf9aa307bde9a64ec8af5b8271a6faa57c83dae10dbe4a25\": container with ID starting with 29eccdf158891ddcdf9aa307bde9a64ec8af5b8271a6faa57c83dae10dbe4a25 not found: ID does not exist" containerID="29eccdf158891ddcdf9aa307bde9a64ec8af5b8271a6faa57c83dae10dbe4a25"
Sep 14 01:52:31.653192 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:31.653139    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29eccdf158891ddcdf9aa307bde9a64ec8af5b8271a6faa57c83dae10dbe4a25"} err="failed to get container status \"29eccdf158891ddcdf9aa307bde9a64ec8af5b8271a6faa57c83dae10dbe4a25\": rpc error: code = NotFound desc = could not find container \"29eccdf158891ddcdf9aa307bde9a64ec8af5b8271a6faa57c83dae10dbe4a25\": container with ID starting with 29eccdf158891ddcdf9aa307bde9a64ec8af5b8271a6faa57c83dae10dbe4a25 not found: ID does not exist"
Sep 14 01:52:32.807464 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:32.807438    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-90142c"
Sep 14 01:52:32.933848 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:32.933753    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql6rf\" (UniqueName: \"kubernetes.io/projected/52049aad-5081-4d84-aeb5-c1db17c79f6d-kube-api-access-ql6rf\") pod \"52049aad-5081-4d84-aeb5-c1db17c79f6d\" (UID: \"52049aad-5081-4d84-aeb5-c1db17c79f6d\") "
Sep 14 01:52:32.935950 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:32.935903    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52049aad-5081-4d84-aeb5-c1db17c79f6d-kube-api-access-ql6rf" (OuterVolumeSpecName: "kube-api-access-ql6rf") pod "52049aad-5081-4d84-aeb5-c1db17c79f6d" (UID: "52049aad-5081-4d84-aeb5-c1db17c79f6d"). InnerVolumeSpecName "kube-api-access-ql6rf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:52:33.035425 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:33.035377    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql6rf\" (UniqueName: \"kubernetes.io/projected/52049aad-5081-4d84-aeb5-c1db17c79f6d-kube-api-access-ql6rf\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:52:33.054982 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:33.054949    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52049aad-5081-4d84-aeb5-c1db17c79f6d" path="/var/lib/kubelet/pods/52049aad-5081-4d84-aeb5-c1db17c79f6d/volumes"
Sep 14 01:52:33.658026 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:33.657994    2581 generic.go:356] "Generic (PLEG): container finished" podID="52049aad-5081-4d84-aeb5-c1db17c79f6d" containerID="7827b9dca520827460d9051cefc6664cb8cff8c0dfe629fa22013ddc0be218ed" exitCode=137
Sep 14 01:52:33.658247 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:33.658037    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7443-90142c"
Sep 14 01:52:33.658247 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:33.658099    2581 scope.go:117] "RemoveContainer" containerID="7827b9dca520827460d9051cefc6664cb8cff8c0dfe629fa22013ddc0be218ed"
Sep 14 01:52:33.667392 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:33.667375    2581 scope.go:117] "RemoveContainer" containerID="7827b9dca520827460d9051cefc6664cb8cff8c0dfe629fa22013ddc0be218ed"
Sep 14 01:52:33.667684 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:52:33.667663    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7827b9dca520827460d9051cefc6664cb8cff8c0dfe629fa22013ddc0be218ed\": container with ID starting with 7827b9dca520827460d9051cefc6664cb8cff8c0dfe629fa22013ddc0be218ed not found: ID does not exist" containerID="7827b9dca520827460d9051cefc6664cb8cff8c0dfe629fa22013ddc0be218ed"
Sep 14 01:52:33.667759 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:33.667692    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7827b9dca520827460d9051cefc6664cb8cff8c0dfe629fa22013ddc0be218ed"} err="failed to get container status \"7827b9dca520827460d9051cefc6664cb8cff8c0dfe629fa22013ddc0be218ed\": rpc error: code = NotFound desc = could not find container \"7827b9dca520827460d9051cefc6664cb8cff8c0dfe629fa22013ddc0be218ed\": container with ID starting with 7827b9dca520827460d9051cefc6664cb8cff8c0dfe629fa22013ddc0be218ed not found: ID does not exist"
Sep 14 01:52:37.117532 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.117496    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"]
Sep 14 01:52:37.117950 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.117936    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52049aad-5081-4d84-aeb5-c1db17c79f6d" containerName="test-curl-7443-90142c"
Sep 14 01:52:37.118014 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.117952    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="52049aad-5081-4d84-aeb5-c1db17c79f6d" containerName="test-curl-7443-90142c"
Sep 14 01:52:37.118061 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.118051    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="52049aad-5081-4d84-aeb5-c1db17c79f6d" containerName="test-curl-7443-90142c"
Sep 14 01:52:37.122820 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.122796    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:37.124767 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.124743    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-000f34a9-serving-cert\""
Sep 14 01:52:37.130529 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.130504    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"]
Sep 14 01:52:37.176971 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.176906    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-openshift-service-ca\") pod \"maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:37.177159 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.177019    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbx9l\" (UniqueName: \"kubernetes.io/projected/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-kube-api-access-fbx9l\") pod \"maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:37.177159 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.177071    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-metrics-certs\") pod \"maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:37.177159 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.177151    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-maas-api-tls\") pod \"maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:37.278645 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.278605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-openshift-service-ca\") pod \"maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:37.278846 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.278670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbx9l\" (UniqueName: \"kubernetes.io/projected/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-kube-api-access-fbx9l\") pod \"maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:37.278846 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.278697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-metrics-certs\") pod \"maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:37.278846 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.278753    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-maas-api-tls\") pod \"maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:37.279427 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.279394    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-openshift-service-ca\") pod \"maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:37.281307 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.281277    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-metrics-certs\") pod \"maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:37.281468 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.281447    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-maas-api-tls\") pod \"maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:37.285528 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.285505    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbx9l\" (UniqueName: \"kubernetes.io/projected/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-kube-api-access-fbx9l\") pod \"maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:37.437265 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.437176    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:37.713081 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:37.713048    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"]
Sep 14 01:52:37.715271 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:52:37.715234    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4a8e0e_8477_4dbc_9a44_b943638b59fa.slice/crio-8d76becbc7c59f7b626ee590a3ef295743612cdb0da64031bd9774927b9db682 WatchSource:0}: Error finding container 8d76becbc7c59f7b626ee590a3ef295743612cdb0da64031bd9774927b9db682: Status 404 returned error can't find the container with id 8d76becbc7c59f7b626ee590a3ef295743612cdb0da64031bd9774927b9db682
Sep 14 01:52:38.691996 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:38.691960    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp" event={"ID":"eb4a8e0e-8477-4dbc-9a44-b943638b59fa","Type":"ContainerStarted","Data":"f54c460b46f89cbc94d2867a0ec1451deb89e6d5d1e3f59104a6df51bdc0509d"}
Sep 14 01:52:38.692385 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:38.692018    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp" event={"ID":"eb4a8e0e-8477-4dbc-9a44-b943638b59fa","Type":"ContainerStarted","Data":"8d76becbc7c59f7b626ee590a3ef295743612cdb0da64031bd9774927b9db682"}
Sep 14 01:52:38.692385 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:38.692096    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:38.707227 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:38.707176    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp" podStartSLOduration=1.269726079 podStartE2EDuration="1.707159855s" totalImagesPullingTime="437.433776ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:52:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:52:38.705626214 +0000 UTC m=+1318.186077035" watchObservedRunningTime="2026-09-14 01:52:38.707159855 +0000 UTC m=+1318.187610728"
Sep 14 01:52:44.704030 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:44.703996    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:52:56.567478 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:52:56.567439    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:53:08.096800 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:08.096702    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"]
Sep 14 01:53:19.364465 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:19.364425    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:53:23.119584 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.119526    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5" podUID="d72a2797-4017-460f-83f6-6b8444e5e9df" containerName="main" containerID="cri-o://e7608ca28122f8bdfd5fe116416252479319b33f2a54af2129ec16daa040c46e" gracePeriod=45
Sep 14 01:53:23.387691 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.387657    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:53:23.585033 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.584984    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzwq9\" (UniqueName: \"kubernetes.io/projected/d72a2797-4017-460f-83f6-6b8444e5e9df-kube-api-access-nzwq9\") pod \"d72a2797-4017-460f-83f6-6b8444e5e9df\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") "
Sep 14 01:53:23.585033 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.585047    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-tmp-dir\") pod \"d72a2797-4017-460f-83f6-6b8444e5e9df\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") "
Sep 14 01:53:23.585302 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.585085    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-home\") pod \"d72a2797-4017-460f-83f6-6b8444e5e9df\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") "
Sep 14 01:53:23.585302 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.585178    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-dshm\") pod \"d72a2797-4017-460f-83f6-6b8444e5e9df\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") "
Sep 14 01:53:23.585302 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.585209    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-model-cache\") pod \"d72a2797-4017-460f-83f6-6b8444e5e9df\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") "
Sep 14 01:53:23.585302 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.585226    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d72a2797-4017-460f-83f6-6b8444e5e9df-tls-certs\") pod \"d72a2797-4017-460f-83f6-6b8444e5e9df\" (UID: \"d72a2797-4017-460f-83f6-6b8444e5e9df\") "
Sep 14 01:53:23.585515 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.585365    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d72a2797-4017-460f-83f6-6b8444e5e9df" (UID: "d72a2797-4017-460f-83f6-6b8444e5e9df"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:53:23.585515 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.585462    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-home" (OuterVolumeSpecName: "home") pod "d72a2797-4017-460f-83f6-6b8444e5e9df" (UID: "d72a2797-4017-460f-83f6-6b8444e5e9df"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:53:23.585515 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.585495    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-model-cache" (OuterVolumeSpecName: "model-cache") pod "d72a2797-4017-460f-83f6-6b8444e5e9df" (UID: "d72a2797-4017-460f-83f6-6b8444e5e9df"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:53:23.585652 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.585593    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:23.585652 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.585612    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:23.585652 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.585631    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:23.587371 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.587348    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-dshm" (OuterVolumeSpecName: "dshm") pod "d72a2797-4017-460f-83f6-6b8444e5e9df" (UID: "d72a2797-4017-460f-83f6-6b8444e5e9df"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:53:23.587453 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.587419    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d72a2797-4017-460f-83f6-6b8444e5e9df-kube-api-access-nzwq9" (OuterVolumeSpecName: "kube-api-access-nzwq9") pod "d72a2797-4017-460f-83f6-6b8444e5e9df" (UID: "d72a2797-4017-460f-83f6-6b8444e5e9df"). InnerVolumeSpecName "kube-api-access-nzwq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:53:23.587453 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.587424    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d72a2797-4017-460f-83f6-6b8444e5e9df-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d72a2797-4017-460f-83f6-6b8444e5e9df" (UID: "d72a2797-4017-460f-83f6-6b8444e5e9df"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:53:23.686522 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.686413    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d72a2797-4017-460f-83f6-6b8444e5e9df-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:23.686522 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.686457    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d72a2797-4017-460f-83f6-6b8444e5e9df-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:23.686522 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.686469    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzwq9\" (UniqueName: \"kubernetes.io/projected/d72a2797-4017-460f-83f6-6b8444e5e9df-kube-api-access-nzwq9\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:23.711608 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.711568    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"]
Sep 14 01:53:23.712039 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.712025    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d72a2797-4017-460f-83f6-6b8444e5e9df" containerName="main"
Sep 14 01:53:23.712090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.712041    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d72a2797-4017-460f-83f6-6b8444e5e9df" containerName="main"
Sep 14 01:53:23.712129 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.712112    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d72a2797-4017-460f-83f6-6b8444e5e9df" containerName="main"
Sep 14 01:53:23.715350 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.715328    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.717490 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.717466    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-06e7b2\"/\"openshift-service-ca.crt\""
Sep 14 01:53:23.718098 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.718063    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-06e7b2\"/\"default-dockercfg-cw7rc\""
Sep 14 01:53:23.718234 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.718106    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-06e7b2\"/\"noten-model-06e7b2-kserve-self-signed-certs\""
Sep 14 01:53:23.718234 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.718064    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-06e7b2\"/\"kube-root-ca.crt\""
Sep 14 01:53:23.724558 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.724510    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"]
Sep 14 01:53:23.887999 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.887958    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffb6184c-1afa-4c6d-b161-3f8e771c544e-tls-certs\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.888184 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.888081    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-model-cache\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.888184 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.888112    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-dshm\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.888184 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.888154    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-home\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.888291 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.888243    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-tmp-dir\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.888291 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.888279    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm7n5\" (UniqueName: \"kubernetes.io/projected/ffb6184c-1afa-4c6d-b161-3f8e771c544e-kube-api-access-nm7n5\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.967443 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.967409    2581 generic.go:356] "Generic (PLEG): container finished" podID="d72a2797-4017-460f-83f6-6b8444e5e9df" containerID="e7608ca28122f8bdfd5fe116416252479319b33f2a54af2129ec16daa040c46e" exitCode=0
Sep 14 01:53:23.967607 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.967472    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"
Sep 14 01:53:23.967607 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.967495    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5" event={"ID":"d72a2797-4017-460f-83f6-6b8444e5e9df","Type":"ContainerDied","Data":"e7608ca28122f8bdfd5fe116416252479319b33f2a54af2129ec16daa040c46e"}
Sep 14 01:53:23.967607 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.967543    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5" event={"ID":"d72a2797-4017-460f-83f6-6b8444e5e9df","Type":"ContainerDied","Data":"f18754fe89f305098e8e0749df9216d95c1a086ba6e2dbe5f78decdab5ef092f"}
Sep 14 01:53:23.967607 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.967564    2581 scope.go:117] "RemoveContainer" containerID="e7608ca28122f8bdfd5fe116416252479319b33f2a54af2129ec16daa040c46e"
Sep 14 01:53:23.977306 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.977285    2581 scope.go:117] "RemoveContainer" containerID="e7608ca28122f8bdfd5fe116416252479319b33f2a54af2129ec16daa040c46e"
Sep 14 01:53:23.977595 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:53:23.977573    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7608ca28122f8bdfd5fe116416252479319b33f2a54af2129ec16daa040c46e\": container with ID starting with e7608ca28122f8bdfd5fe116416252479319b33f2a54af2129ec16daa040c46e not found: ID does not exist" containerID="e7608ca28122f8bdfd5fe116416252479319b33f2a54af2129ec16daa040c46e"
Sep 14 01:53:23.977668 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.977602    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7608ca28122f8bdfd5fe116416252479319b33f2a54af2129ec16daa040c46e"} err="failed to get container status \"e7608ca28122f8bdfd5fe116416252479319b33f2a54af2129ec16daa040c46e\": rpc error: code = NotFound desc = could not find container \"e7608ca28122f8bdfd5fe116416252479319b33f2a54af2129ec16daa040c46e\": container with ID starting with e7608ca28122f8bdfd5fe116416252479319b33f2a54af2129ec16daa040c46e not found: ID does not exist"
Sep 14 01:53:23.987987 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.987963    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"]
Sep 14 01:53:23.989253 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.989223    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-tmp-dir\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.989407 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.989269    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm7n5\" (UniqueName: \"kubernetes.io/projected/ffb6184c-1afa-4c6d-b161-3f8e771c544e-kube-api-access-nm7n5\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.989407 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.989362    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffb6184c-1afa-4c6d-b161-3f8e771c544e-tls-certs\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.989524 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.989449    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-model-cache\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.989524 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.989472    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-dshm\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.989524 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.989516    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-home\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.990033 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.989999    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-home\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.990159 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.990052    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-tmp-dir\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.990159 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.990077    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-model-cache\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.991998 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.991976    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-dshm\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.992286 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.992266    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffb6184c-1afa-4c6d-b161-3f8e771c544e-tls-certs\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:23.992453 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.992439    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-a36bf3/test-model-a36bf3-kserve-555dd98c47-85pv5"]
Sep 14 01:53:23.996460 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:23.996444    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm7n5\" (UniqueName: \"kubernetes.io/projected/ffb6184c-1afa-4c6d-b161-3f8e771c544e-kube-api-access-nm7n5\") pod \"noten-model-06e7b2-kserve-5b5c86d69f-8zg4v\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") " pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:24.028398 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:24.028357    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:24.302588 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:24.302435    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"]
Sep 14 01:53:24.305356 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:53:24.305326    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podffb6184c_1afa_4c6d_b161_3f8e771c544e.slice/crio-6dc2ca802eeff38683b6fa03f2645cfcb4a3c1c7b82735bece045fd85def6f7b WatchSource:0}: Error finding container 6dc2ca802eeff38683b6fa03f2645cfcb4a3c1c7b82735bece045fd85def6f7b: Status 404 returned error can't find the container with id 6dc2ca802eeff38683b6fa03f2645cfcb4a3c1c7b82735bece045fd85def6f7b
Sep 14 01:53:24.976704 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:24.976670    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v" event={"ID":"ffb6184c-1afa-4c6d-b161-3f8e771c544e","Type":"ContainerStarted","Data":"dd80fb87218a03002fcf79653f3400741fbe8c53dcda0bb08f8caa017ef6e5e5"}
Sep 14 01:53:24.976704 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:24.976712    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v" event={"ID":"ffb6184c-1afa-4c6d-b161-3f8e771c544e","Type":"ContainerStarted","Data":"6dc2ca802eeff38683b6fa03f2645cfcb4a3c1c7b82735bece045fd85def6f7b"}
Sep 14 01:53:24.994316 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:24.994262    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v" podStartSLOduration=1.994246545 podStartE2EDuration="1.994246545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:53:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:53:24.991705578 +0000 UTC m=+1364.472156410" watchObservedRunningTime="2026-09-14 01:53:24.994246545 +0000 UTC m=+1364.474697365"
Sep 14 01:53:25.055160 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:25.055126    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d72a2797-4017-460f-83f6-6b8444e5e9df" path="/var/lib/kubelet/pods/d72a2797-4017-460f-83f6-6b8444e5e9df/volumes"
Sep 14 01:53:34.029352 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:34.029319    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:34.029994 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:34.029390    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:34.042204 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:34.042180    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:35.054707 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:35.054674    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:53:37.316940 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:37.313847    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"]
Sep 14 01:53:45.903788 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:45.903748    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"]
Sep 14 01:53:45.904294 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:45.904012    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c" podUID="c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562" containerName="maas-api" containerID="cri-o://f7970f61dc4c3eee8694de5d2fbcb4afb599cb87900d656178401f7e6798e2e0" gracePeriod=30
Sep 14 01:53:46.106538 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:46.106504    2581 generic.go:356] "Generic (PLEG): container finished" podID="c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562" containerID="f7970f61dc4c3eee8694de5d2fbcb4afb599cb87900d656178401f7e6798e2e0" exitCode=0
Sep 14 01:53:46.106698 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:46.106579    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c" event={"ID":"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562","Type":"ContainerDied","Data":"f7970f61dc4c3eee8694de5d2fbcb4afb599cb87900d656178401f7e6798e2e0"}
Sep 14 01:53:46.162179 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:46.162116    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:53:46.192927 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:46.192885    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dhq8\" (UniqueName: \"kubernetes.io/projected/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-kube-api-access-7dhq8\") pod \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") "
Sep 14 01:53:46.193112 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:46.192969    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-openshift-service-ca\") pod \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") "
Sep 14 01:53:46.193112 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:46.192990    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-metrics-certs\") pod \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") "
Sep 14 01:53:46.193112 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:46.193047    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-maas-api-tls\") pod \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\" (UID: \"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562\") "
Sep 14 01:53:46.193382 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:46.193356    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562" (UID: "c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:53:46.193511 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:46.193495    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:46.195401 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:46.195367    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-kube-api-access-7dhq8" (OuterVolumeSpecName: "kube-api-access-7dhq8") pod "c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562" (UID: "c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562"). InnerVolumeSpecName "kube-api-access-7dhq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:53:46.195401 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:46.195371    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562" (UID: "c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:53:46.195597 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:46.195397    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562" (UID: "c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:53:46.294810 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:46.294766    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dhq8\" (UniqueName: \"kubernetes.io/projected/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-kube-api-access-7dhq8\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:46.294810 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:46.294801    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:46.294810 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:46.294814    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:47.114196 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:47.114162    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"
Sep 14 01:53:47.114692 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:47.114166    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c" event={"ID":"c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562","Type":"ContainerDied","Data":"8ea9da3c3dbac4b071febd0db0c24a24c880cba2ac0962cd3bb9283a7ce97b1c"}
Sep 14 01:53:47.114692 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:47.114309    2581 scope.go:117] "RemoveContainer" containerID="f7970f61dc4c3eee8694de5d2fbcb4afb599cb87900d656178401f7e6798e2e0"
Sep 14 01:53:47.139020 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:47.138988    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"]
Sep 14 01:53:47.144736 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:47.144703    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-321811-ffd6c64b7-59n6c"]
Sep 14 01:53:47.664687 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:47.664647    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:53:49.054401 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:49.054365    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562" path="/var/lib/kubelet/pods/c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562/volumes"
Sep 14 01:53:52.342630 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.342568    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk" podUID="7502f0c1-553d-4d21-ab16-dbb00afa7f62" containerName="main" containerID="cri-o://0cac9a534ef987d339f4034d5d50d62fcc837f8854f2e320882b3614789c72cd" gracePeriod=45
Sep 14 01:53:52.602573 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.602506    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:53:52.663509 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.663474    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64vxn\" (UniqueName: \"kubernetes.io/projected/7502f0c1-553d-4d21-ab16-dbb00afa7f62-kube-api-access-64vxn\") pod \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") "
Sep 14 01:53:52.663509 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.663518    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-model-cache\") pod \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") "
Sep 14 01:53:52.663807 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.663605    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-tmp-dir\") pod \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") "
Sep 14 01:53:52.663807 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.663639    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7502f0c1-553d-4d21-ab16-dbb00afa7f62-tls-certs\") pod \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") "
Sep 14 01:53:52.663807 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.663693    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-home\") pod \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") "
Sep 14 01:53:52.663807 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.663762    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-dshm\") pod \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\" (UID: \"7502f0c1-553d-4d21-ab16-dbb00afa7f62\") "
Sep 14 01:53:52.664013 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.663842    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-model-cache" (OuterVolumeSpecName: "model-cache") pod "7502f0c1-553d-4d21-ab16-dbb00afa7f62" (UID: "7502f0c1-553d-4d21-ab16-dbb00afa7f62"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:53:52.664013 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.663871    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7502f0c1-553d-4d21-ab16-dbb00afa7f62" (UID: "7502f0c1-553d-4d21-ab16-dbb00afa7f62"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:53:52.664112 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.664050    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-home" (OuterVolumeSpecName: "home") pod "7502f0c1-553d-4d21-ab16-dbb00afa7f62" (UID: "7502f0c1-553d-4d21-ab16-dbb00afa7f62"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:53:52.664228 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.664213    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:52.664285 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.664234    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:52.664285 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.664245    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:52.665969 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.665943    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-dshm" (OuterVolumeSpecName: "dshm") pod "7502f0c1-553d-4d21-ab16-dbb00afa7f62" (UID: "7502f0c1-553d-4d21-ab16-dbb00afa7f62"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:53:52.666089 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.665967    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7502f0c1-553d-4d21-ab16-dbb00afa7f62-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7502f0c1-553d-4d21-ab16-dbb00afa7f62" (UID: "7502f0c1-553d-4d21-ab16-dbb00afa7f62"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:53:52.666089 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.666020    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7502f0c1-553d-4d21-ab16-dbb00afa7f62-kube-api-access-64vxn" (OuterVolumeSpecName: "kube-api-access-64vxn") pod "7502f0c1-553d-4d21-ab16-dbb00afa7f62" (UID: "7502f0c1-553d-4d21-ab16-dbb00afa7f62"). InnerVolumeSpecName "kube-api-access-64vxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:53:52.768331 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.765625    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7502f0c1-553d-4d21-ab16-dbb00afa7f62-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:52.768331 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.765668    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7502f0c1-553d-4d21-ab16-dbb00afa7f62-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:52.768331 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:52.765693    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64vxn\" (UniqueName: \"kubernetes.io/projected/7502f0c1-553d-4d21-ab16-dbb00afa7f62-kube-api-access-64vxn\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:53:53.152025 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:53.151982    2581 generic.go:356] "Generic (PLEG): container finished" podID="7502f0c1-553d-4d21-ab16-dbb00afa7f62" containerID="0cac9a534ef987d339f4034d5d50d62fcc837f8854f2e320882b3614789c72cd" exitCode=0
Sep 14 01:53:53.152229 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:53.152048    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"
Sep 14 01:53:53.152229 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:53.152065    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk" event={"ID":"7502f0c1-553d-4d21-ab16-dbb00afa7f62","Type":"ContainerDied","Data":"0cac9a534ef987d339f4034d5d50d62fcc837f8854f2e320882b3614789c72cd"}
Sep 14 01:53:53.152229 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:53.152123    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk" event={"ID":"7502f0c1-553d-4d21-ab16-dbb00afa7f62","Type":"ContainerDied","Data":"de691f661337abc3a15d4117ca4ffff1f090c4c51897ef17dfd8d564b2427ace"}
Sep 14 01:53:53.152229 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:53.152145    2581 scope.go:117] "RemoveContainer" containerID="0cac9a534ef987d339f4034d5d50d62fcc837f8854f2e320882b3614789c72cd"
Sep 14 01:53:53.161674 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:53.161653    2581 scope.go:117] "RemoveContainer" containerID="0cac9a534ef987d339f4034d5d50d62fcc837f8854f2e320882b3614789c72cd"
Sep 14 01:53:53.161997 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:53:53.161975    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cac9a534ef987d339f4034d5d50d62fcc837f8854f2e320882b3614789c72cd\": container with ID starting with 0cac9a534ef987d339f4034d5d50d62fcc837f8854f2e320882b3614789c72cd not found: ID does not exist" containerID="0cac9a534ef987d339f4034d5d50d62fcc837f8854f2e320882b3614789c72cd"
Sep 14 01:53:53.162050 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:53.162012    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cac9a534ef987d339f4034d5d50d62fcc837f8854f2e320882b3614789c72cd"} err="failed to get container status \"0cac9a534ef987d339f4034d5d50d62fcc837f8854f2e320882b3614789c72cd\": rpc error: code = NotFound desc = could not find container \"0cac9a534ef987d339f4034d5d50d62fcc837f8854f2e320882b3614789c72cd\": container with ID starting with 0cac9a534ef987d339f4034d5d50d62fcc837f8854f2e320882b3614789c72cd not found: ID does not exist"
Sep 14 01:53:53.167340 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:53.167311    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"]
Sep 14 01:53:53.170663 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:53.170638    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-321811/test-model-321811-kserve-754db89857-c6ggk"]
Sep 14 01:53:55.055159 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:55.055125    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7502f0c1-553d-4d21-ab16-dbb00afa7f62" path="/var/lib/kubelet/pods/7502f0c1-553d-4d21-ab16-dbb00afa7f62/volumes"
Sep 14 01:53:56.799350 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.799308    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"]
Sep 14 01:53:56.799770 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.799756    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562" containerName="maas-api"
Sep 14 01:53:56.799815 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.799772    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562" containerName="maas-api"
Sep 14 01:53:56.799815 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.799783    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7502f0c1-553d-4d21-ab16-dbb00afa7f62" containerName="main"
Sep 14 01:53:56.799815 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.799788    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7502f0c1-553d-4d21-ab16-dbb00afa7f62" containerName="main"
Sep 14 01:53:56.799938 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.799872    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7502f0c1-553d-4d21-ab16-dbb00afa7f62" containerName="main"
Sep 14 01:53:56.799938 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.799880    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2fc2b86-9eb8-4d3c-b27f-3c3fbb90d562" containerName="maas-api"
Sep 14 01:53:56.803234 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.803216    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:56.805124 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.805098    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-99c616\"/\"distinct-2-w0-99c616-kserve-self-signed-certs\""
Sep 14 01:53:56.810262 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.810232    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"]
Sep 14 01:53:56.907979 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.907900    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-dshm\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:56.908173 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.908052    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-model-cache\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:56.908173 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.908151    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cw79\" (UniqueName: \"kubernetes.io/projected/74e30abc-2964-475a-a83d-33cef503d42b-kube-api-access-6cw79\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:56.908258 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.908188    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-home\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:56.908258 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.908237    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74e30abc-2964-475a-a83d-33cef503d42b-tls-certs\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:56.908328 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:56.908260    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-tmp-dir\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:57.009389 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:57.009348    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-tmp-dir\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:57.009573 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:57.009429    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-dshm\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:57.009573 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:57.009480    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-model-cache\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:57.009573 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:57.009565    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cw79\" (UniqueName: \"kubernetes.io/projected/74e30abc-2964-475a-a83d-33cef503d42b-kube-api-access-6cw79\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:57.009683 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:57.009610    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-home\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:57.009683 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:57.009642    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74e30abc-2964-475a-a83d-33cef503d42b-tls-certs\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:57.009847 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:57.009827    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-tmp-dir\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:57.010007 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:57.009978    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-model-cache\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:57.010145 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:57.009990    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-home\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:57.011758 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:57.011736    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-dshm\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:57.012195 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:57.012174    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74e30abc-2964-475a-a83d-33cef503d42b-tls-certs\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:57.016564 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:57.016535    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cw79\" (UniqueName: \"kubernetes.io/projected/74e30abc-2964-475a-a83d-33cef503d42b-kube-api-access-6cw79\") pod \"distinct-2-w0-99c616-kserve-5497c76956-pdfkt\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") " pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:57.116319 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:57.116223    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:53:57.389520 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:57.389490    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"]
Sep 14 01:53:57.391089 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:53:57.391060    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74e30abc_2964_475a_a83d_33cef503d42b.slice/crio-c5499a24f3347c9b860e14bb880aed9fcd19d7a162a9221767a6fa9bd6a037ab WatchSource:0}: Error finding container c5499a24f3347c9b860e14bb880aed9fcd19d7a162a9221767a6fa9bd6a037ab: Status 404 returned error can't find the container with id c5499a24f3347c9b860e14bb880aed9fcd19d7a162a9221767a6fa9bd6a037ab
Sep 14 01:53:58.187153 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:58.187112    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt" event={"ID":"74e30abc-2964-475a-a83d-33cef503d42b","Type":"ContainerStarted","Data":"7b4d8badff54c0666beeb30cc0224ba1246945d49673b580913e5dfb7892a306"}
Sep 14 01:53:58.187553 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:58.187167    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt" event={"ID":"74e30abc-2964-475a-a83d-33cef503d42b","Type":"ContainerStarted","Data":"c5499a24f3347c9b860e14bb880aed9fcd19d7a162a9221767a6fa9bd6a037ab"}
Sep 14 01:53:58.205104 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:53:58.205041    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt" podStartSLOduration=2.205023883 podStartE2EDuration="2.205023883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:53:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:53:58.202651132 +0000 UTC m=+1397.683101953" watchObservedRunningTime="2026-09-14 01:53:58.205023883 +0000 UTC m=+1397.685474704"
Sep 14 01:54:01.087773 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:01.087738    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"]
Sep 14 01:54:04.804209 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:04.804175    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"]
Sep 14 01:54:04.808187 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:04.808161    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:04.810188 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:04.810159    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-5d3ac7-serving-cert\""
Sep 14 01:54:04.819282 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:04.819257    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"]
Sep 14 01:54:04.992981 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:04.992908    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0f292d1-9579-43e8-9c16-752da3c80f30-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:04.993227 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:04.993061    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0f292d1-9579-43e8-9c16-752da3c80f30-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:04.993227 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:04.993100    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68fhd\" (UniqueName: \"kubernetes.io/projected/b0f292d1-9579-43e8-9c16-752da3c80f30-kube-api-access-68fhd\") pod \"maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:04.993227 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:04.993137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0f292d1-9579-43e8-9c16-752da3c80f30-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:05.094719 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:05.094615    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0f292d1-9579-43e8-9c16-752da3c80f30-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:05.094719 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:05.094678    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68fhd\" (UniqueName: \"kubernetes.io/projected/b0f292d1-9579-43e8-9c16-752da3c80f30-kube-api-access-68fhd\") pod \"maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:05.094719 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:05.094717    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0f292d1-9579-43e8-9c16-752da3c80f30-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:05.095084 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:05.094781    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0f292d1-9579-43e8-9c16-752da3c80f30-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:05.095084 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:54:05.094876    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-serving-cert: secret "maas-api-e2e-shared-a-w3-5d3ac7-serving-cert" not found
Sep 14 01:54:05.095084 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:54:05.094982    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0f292d1-9579-43e8-9c16-752da3c80f30-maas-api-tls podName:b0f292d1-9579-43e8-9c16-752da3c80f30 nodeName:}" failed. No retries permitted until 2026-09-14 01:54:05.594959185 +0000 UTC m=+1405.075409987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b0f292d1-9579-43e8-9c16-752da3c80f30-maas-api-tls") pod "maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4" (UID: "b0f292d1-9579-43e8-9c16-752da3c80f30") : secret "maas-api-e2e-shared-a-w3-5d3ac7-serving-cert" not found
Sep 14 01:54:05.095350 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:05.095331    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0f292d1-9579-43e8-9c16-752da3c80f30-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:05.097385 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:05.097359    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0f292d1-9579-43e8-9c16-752da3c80f30-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:05.102374 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:05.102348    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68fhd\" (UniqueName: \"kubernetes.io/projected/b0f292d1-9579-43e8-9c16-752da3c80f30-kube-api-access-68fhd\") pod \"maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:05.601440 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:05.601385    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0f292d1-9579-43e8-9c16-752da3c80f30-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:05.603888 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:05.603861    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0f292d1-9579-43e8-9c16-752da3c80f30-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:05.722707 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:05.722676    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:05.998781 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:05.998754    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"]
Sep 14 01:54:06.001073 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:54:06.001043    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f292d1_9579_43e8_9c16_752da3c80f30.slice/crio-8dd1ded9d64d64520a085cec029e38045af831601876434b4ccca1864d500b8b WatchSource:0}: Error finding container 8dd1ded9d64d64520a085cec029e38045af831601876434b4ccca1864d500b8b: Status 404 returned error can't find the container with id 8dd1ded9d64d64520a085cec029e38045af831601876434b4ccca1864d500b8b
Sep 14 01:54:06.238166 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:06.238131    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4" event={"ID":"b0f292d1-9579-43e8-9c16-752da3c80f30","Type":"ContainerStarted","Data":"8dd1ded9d64d64520a085cec029e38045af831601876434b4ccca1864d500b8b"}
Sep 14 01:54:07.117212 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:07.117179    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:54:07.117212 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:07.117219    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:54:07.130187 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:07.130152    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:54:07.245543 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:07.245496    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4" event={"ID":"b0f292d1-9579-43e8-9c16-752da3c80f30","Type":"ContainerStarted","Data":"b10b7be5292a49910c2919937b98c79c747814f26ce5a6c36861c1972ecb4242"}
Sep 14 01:54:07.245767 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:07.245749    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:07.257305 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:07.257276    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:54:07.262504 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:07.262454    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4" podStartSLOduration=2.647334866 podStartE2EDuration="3.262437849s" totalImagesPullingTime="615.102983ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:54:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:54:07.260518298 +0000 UTC m=+1406.740969120" watchObservedRunningTime="2026-09-14 01:54:07.262437849 +0000 UTC m=+1406.742888669"
Sep 14 01:54:08.858375 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:08.858335    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:54:13.257187 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:13.257156    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 01:54:16.107596 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.107527    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v" podUID="ffb6184c-1afa-4c6d-b161-3f8e771c544e" containerName="main" containerID="cri-o://dd80fb87218a03002fcf79653f3400741fbe8c53dcda0bb08f8caa017ef6e5e5" gracePeriod=45
Sep 14 01:54:16.306239 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.306208    2581 generic.go:356] "Generic (PLEG): container finished" podID="ffb6184c-1afa-4c6d-b161-3f8e771c544e" containerID="dd80fb87218a03002fcf79653f3400741fbe8c53dcda0bb08f8caa017ef6e5e5" exitCode=0
Sep 14 01:54:16.306239 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.306246    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v" event={"ID":"ffb6184c-1afa-4c6d-b161-3f8e771c544e","Type":"ContainerDied","Data":"dd80fb87218a03002fcf79653f3400741fbe8c53dcda0bb08f8caa017ef6e5e5"}
Sep 14 01:54:16.368192 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.368121    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:54:16.410829 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.410783    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nm7n5\" (UniqueName: \"kubernetes.io/projected/ffb6184c-1afa-4c6d-b161-3f8e771c544e-kube-api-access-nm7n5\") pod \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") "
Sep 14 01:54:16.410829 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.410831    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffb6184c-1afa-4c6d-b161-3f8e771c544e-tls-certs\") pod \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") "
Sep 14 01:54:16.411116 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.410886    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-model-cache\") pod \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") "
Sep 14 01:54:16.411116 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.410973    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-tmp-dir\") pod \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") "
Sep 14 01:54:16.411116 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.411011    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-home\") pod \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") "
Sep 14 01:54:16.411116 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.411050    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-dshm\") pod \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\" (UID: \"ffb6184c-1afa-4c6d-b161-3f8e771c544e\") "
Sep 14 01:54:16.411338 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.411270    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-model-cache" (OuterVolumeSpecName: "model-cache") pod "ffb6184c-1afa-4c6d-b161-3f8e771c544e" (UID: "ffb6184c-1afa-4c6d-b161-3f8e771c544e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:54:16.411338 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.411285    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ffb6184c-1afa-4c6d-b161-3f8e771c544e" (UID: "ffb6184c-1afa-4c6d-b161-3f8e771c544e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:54:16.411537 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.411503    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-home" (OuterVolumeSpecName: "home") pod "ffb6184c-1afa-4c6d-b161-3f8e771c544e" (UID: "ffb6184c-1afa-4c6d-b161-3f8e771c544e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:54:16.411537 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.411519    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:54:16.411704 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.411562    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:54:16.413375 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.413345    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffb6184c-1afa-4c6d-b161-3f8e771c544e-kube-api-access-nm7n5" (OuterVolumeSpecName: "kube-api-access-nm7n5") pod "ffb6184c-1afa-4c6d-b161-3f8e771c544e" (UID: "ffb6184c-1afa-4c6d-b161-3f8e771c544e"). InnerVolumeSpecName "kube-api-access-nm7n5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:54:16.413759 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.413739    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-dshm" (OuterVolumeSpecName: "dshm") pod "ffb6184c-1afa-4c6d-b161-3f8e771c544e" (UID: "ffb6184c-1afa-4c6d-b161-3f8e771c544e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:54:16.413876 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.413847    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffb6184c-1afa-4c6d-b161-3f8e771c544e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ffb6184c-1afa-4c6d-b161-3f8e771c544e" (UID: "ffb6184c-1afa-4c6d-b161-3f8e771c544e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:54:16.512803 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.512751    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:54:16.512803 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.512798    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffb6184c-1afa-4c6d-b161-3f8e771c544e-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:54:16.512803 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.512807    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nm7n5\" (UniqueName: \"kubernetes.io/projected/ffb6184c-1afa-4c6d-b161-3f8e771c544e-kube-api-access-nm7n5\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:54:16.512803 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:16.512819    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffb6184c-1afa-4c6d-b161-3f8e771c544e-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:54:17.313275 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:17.313191    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"
Sep 14 01:54:17.313703 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:17.313191    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v" event={"ID":"ffb6184c-1afa-4c6d-b161-3f8e771c544e","Type":"ContainerDied","Data":"6dc2ca802eeff38683b6fa03f2645cfcb4a3c1c7b82735bece045fd85def6f7b"}
Sep 14 01:54:17.313703 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:17.313315    2581 scope.go:117] "RemoveContainer" containerID="dd80fb87218a03002fcf79653f3400741fbe8c53dcda0bb08f8caa017ef6e5e5"
Sep 14 01:54:17.328782 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:17.328754    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"]
Sep 14 01:54:17.332277 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:17.332251    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-06e7b2/noten-model-06e7b2-kserve-5b5c86d69f-8zg4v"]
Sep 14 01:54:19.054079 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:19.054045    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffb6184c-1afa-4c6d-b161-3f8e771c544e" path="/var/lib/kubelet/pods/ffb6184c-1afa-4c6d-b161-3f8e771c544e/volumes"
Sep 14 01:54:31.059220 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:31.059191    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:54:39.508606 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.508522    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"]
Sep 14 01:54:39.509263 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.509007    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb6184c-1afa-4c6d-b161-3f8e771c544e" containerName="main"
Sep 14 01:54:39.509263 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.509020    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb6184c-1afa-4c6d-b161-3f8e771c544e" containerName="main"
Sep 14 01:54:39.509263 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.509093    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb6184c-1afa-4c6d-b161-3f8e771c544e" containerName="main"
Sep 14 01:54:39.513858 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.513840    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.515884 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.515865    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-8e5935\"/\"e2e-conflict-b-9e04456b-kserve-self-signed-certs\""
Sep 14 01:54:39.520701 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.520678    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"]
Sep 14 01:54:39.547218 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.547186    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbt9l\" (UniqueName: \"kubernetes.io/projected/437af35c-88a2-416f-9a35-669491353849-kube-api-access-tbt9l\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.547385 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.547230    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-model-cache\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.547385 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.547264    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-dshm\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.547385 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.547318    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/437af35c-88a2-416f-9a35-669491353849-tls-certs\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.547385 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.547385    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-tmp-dir\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.547537 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.547414    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-home\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.648851 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.648816    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbt9l\" (UniqueName: \"kubernetes.io/projected/437af35c-88a2-416f-9a35-669491353849-kube-api-access-tbt9l\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.648851 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.648866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-model-cache\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.649169 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.648897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-dshm\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.649169 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.648946    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/437af35c-88a2-416f-9a35-669491353849-tls-certs\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.649169 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.649005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-tmp-dir\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.649169 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.649051    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-home\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.649434 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.649398    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-model-cache\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.649556 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.649440    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-tmp-dir\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.649556 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.649481    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-home\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.651437 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.651416    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-dshm\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.651584 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.651566    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/437af35c-88a2-416f-9a35-669491353849-tls-certs\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.655889 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.655870    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbt9l\" (UniqueName: \"kubernetes.io/projected/437af35c-88a2-416f-9a35-669491353849-kube-api-access-tbt9l\") pod \"e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") " pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:39.826226 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:39.826131    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:40.106737 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:40.106704    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"]
Sep 14 01:54:40.108196 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:54:40.108149    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod437af35c_88a2_416f_9a35_669491353849.slice/crio-354287547ec4e00a2b590441882cf0e13d5594d6a9929601a13c36dbfec8c06b WatchSource:0}: Error finding container 354287547ec4e00a2b590441882cf0e13d5594d6a9929601a13c36dbfec8c06b: Status 404 returned error can't find the container with id 354287547ec4e00a2b590441882cf0e13d5594d6a9929601a13c36dbfec8c06b
Sep 14 01:54:40.452433 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:40.452329    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6" event={"ID":"437af35c-88a2-416f-9a35-669491353849","Type":"ContainerStarted","Data":"2e713f37df264f5465184533cfa088fbf021683fefc16b9857d76bd21fdc1c46"}
Sep 14 01:54:40.452433 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:40.452375    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6" event={"ID":"437af35c-88a2-416f-9a35-669491353849","Type":"ContainerStarted","Data":"354287547ec4e00a2b590441882cf0e13d5594d6a9929601a13c36dbfec8c06b"}
Sep 14 01:54:40.468326 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:40.468273    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6" podStartSLOduration=1.46825554 podStartE2EDuration="1.46825554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:54:40.467736967 +0000 UTC m=+1439.948187789" watchObservedRunningTime="2026-09-14 01:54:40.46825554 +0000 UTC m=+1439.948706361"
Sep 14 01:54:49.826928 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:49.826886    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:49.827430 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:49.826958    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:49.839689 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:49.839659    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:50.527045 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:50.527013    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:54:54.165243 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:54:54.165205    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:55:23.464867 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:23.464827    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:55:30.828720 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:30.828675    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"]
Sep 14 01:55:30.829204 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:30.828965    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp" podUID="eb4a8e0e-8477-4dbc-9a44-b943638b59fa" containerName="maas-api" containerID="cri-o://f54c460b46f89cbc94d2867a0ec1451deb89e6d5d1e3f59104a6df51bdc0509d" gracePeriod=30
Sep 14 01:55:31.087400 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.087336    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:55:31.181578 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.181539    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-metrics-certs\") pod \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") "
Sep 14 01:55:31.181792 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.181599    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-maas-api-tls\") pod \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") "
Sep 14 01:55:31.181792 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.181680    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbx9l\" (UniqueName: \"kubernetes.io/projected/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-kube-api-access-fbx9l\") pod \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") "
Sep 14 01:55:31.181792 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.181755    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-openshift-service-ca\") pod \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\" (UID: \"eb4a8e0e-8477-4dbc-9a44-b943638b59fa\") "
Sep 14 01:55:31.182235 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.182204    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eb4a8e0e-8477-4dbc-9a44-b943638b59fa" (UID: "eb4a8e0e-8477-4dbc-9a44-b943638b59fa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:55:31.183834 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.183800    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "eb4a8e0e-8477-4dbc-9a44-b943638b59fa" (UID: "eb4a8e0e-8477-4dbc-9a44-b943638b59fa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:55:31.183976 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.183825    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eb4a8e0e-8477-4dbc-9a44-b943638b59fa" (UID: "eb4a8e0e-8477-4dbc-9a44-b943638b59fa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:55:31.183976 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.183867    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-kube-api-access-fbx9l" (OuterVolumeSpecName: "kube-api-access-fbx9l") pod "eb4a8e0e-8477-4dbc-9a44-b943638b59fa" (UID: "eb4a8e0e-8477-4dbc-9a44-b943638b59fa"). InnerVolumeSpecName "kube-api-access-fbx9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:55:31.283508 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.283468    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:55:31.283508 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.283501    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbx9l\" (UniqueName: \"kubernetes.io/projected/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-kube-api-access-fbx9l\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:55:31.283508 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.283512    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:55:31.283508 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.283521    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb4a8e0e-8477-4dbc-9a44-b943638b59fa-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:55:31.759891 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.759857    2581 generic.go:356] "Generic (PLEG): container finished" podID="eb4a8e0e-8477-4dbc-9a44-b943638b59fa" containerID="f54c460b46f89cbc94d2867a0ec1451deb89e6d5d1e3f59104a6df51bdc0509d" exitCode=0
Sep 14 01:55:31.760110 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.759903    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp" event={"ID":"eb4a8e0e-8477-4dbc-9a44-b943638b59fa","Type":"ContainerDied","Data":"f54c460b46f89cbc94d2867a0ec1451deb89e6d5d1e3f59104a6df51bdc0509d"}
Sep 14 01:55:31.760110 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.759935    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"
Sep 14 01:55:31.760110 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.760002    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp" event={"ID":"eb4a8e0e-8477-4dbc-9a44-b943638b59fa","Type":"ContainerDied","Data":"8d76becbc7c59f7b626ee590a3ef295743612cdb0da64031bd9774927b9db682"}
Sep 14 01:55:31.760110 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.760028    2581 scope.go:117] "RemoveContainer" containerID="f54c460b46f89cbc94d2867a0ec1451deb89e6d5d1e3f59104a6df51bdc0509d"
Sep 14 01:55:31.769837 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.769820    2581 scope.go:117] "RemoveContainer" containerID="f54c460b46f89cbc94d2867a0ec1451deb89e6d5d1e3f59104a6df51bdc0509d"
Sep 14 01:55:31.770190 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:55:31.770167    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f54c460b46f89cbc94d2867a0ec1451deb89e6d5d1e3f59104a6df51bdc0509d\": container with ID starting with f54c460b46f89cbc94d2867a0ec1451deb89e6d5d1e3f59104a6df51bdc0509d not found: ID does not exist" containerID="f54c460b46f89cbc94d2867a0ec1451deb89e6d5d1e3f59104a6df51bdc0509d"
Sep 14 01:55:31.770278 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.770198    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f54c460b46f89cbc94d2867a0ec1451deb89e6d5d1e3f59104a6df51bdc0509d"} err="failed to get container status \"f54c460b46f89cbc94d2867a0ec1451deb89e6d5d1e3f59104a6df51bdc0509d\": rpc error: code = NotFound desc = could not find container \"f54c460b46f89cbc94d2867a0ec1451deb89e6d5d1e3f59104a6df51bdc0509d\": container with ID starting with f54c460b46f89cbc94d2867a0ec1451deb89e6d5d1e3f59104a6df51bdc0509d not found: ID does not exist"
Sep 14 01:55:31.781199 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.781170    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"]
Sep 14 01:55:31.783787 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:31.783762    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-000f34a9-7899d65d65-8qfrp"]
Sep 14 01:55:33.055244 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:33.055205    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb4a8e0e-8477-4dbc-9a44-b943638b59fa" path="/var/lib/kubelet/pods/eb4a8e0e-8477-4dbc-9a44-b943638b59fa/volumes"
Sep 14 01:55:41.146209 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:41.146180    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 01:55:41.154376 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:41.154351    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 01:55:42.508541 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.508514    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"]
Sep 14 01:55:42.509212 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.509191    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb4a8e0e-8477-4dbc-9a44-b943638b59fa" containerName="maas-api"
Sep 14 01:55:42.509317 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.509215    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb4a8e0e-8477-4dbc-9a44-b943638b59fa" containerName="maas-api"
Sep 14 01:55:42.509374 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.509332    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb4a8e0e-8477-4dbc-9a44-b943638b59fa" containerName="maas-api"
Sep 14 01:55:42.513538 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.513514    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.515692 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.515667    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d3ac7\"/\"default-dockercfg-gw7rs\""
Sep 14 01:55:42.516446 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.516405    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d3ac7\"/\"kube-root-ca.crt\""
Sep 14 01:55:42.516446 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.516434    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d3ac7\"/\"openshift-service-ca.crt\""
Sep 14 01:55:42.516645 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.516562    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d3ac7\"/\"auth-test-model-5d3ac7-kserve-self-signed-certs\""
Sep 14 01:55:42.521689 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.521665    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"]
Sep 14 01:55:42.604139 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.604105    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7p8b\" (UniqueName: \"kubernetes.io/projected/071153ee-6f09-4596-944e-08d1b0fa5a36-kube-api-access-p7p8b\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.604298 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.604212    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-tmp-dir\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.604298 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.604238    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/071153ee-6f09-4596-944e-08d1b0fa5a36-tls-certs\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.604427 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.604351    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-dshm\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.604427 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.604402    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-model-cache\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.604518 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.604441    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-home\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.705048 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.705012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-dshm\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.705048 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.705053    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-model-cache\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.705262 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.705177    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-home\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.705305 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.705273    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7p8b\" (UniqueName: \"kubernetes.io/projected/071153ee-6f09-4596-944e-08d1b0fa5a36-kube-api-access-p7p8b\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.705407 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.705388    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-tmp-dir\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.705458 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.705430    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/071153ee-6f09-4596-944e-08d1b0fa5a36-tls-certs\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.705497 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.705450    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-model-cache\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.705563 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.705504    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-home\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.705717 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.705697    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-tmp-dir\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.707399 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.707372    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-dshm\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.707706 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.707690    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/071153ee-6f09-4596-944e-08d1b0fa5a36-tls-certs\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.712557 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.712530    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7p8b\" (UniqueName: \"kubernetes.io/projected/071153ee-6f09-4596-944e-08d1b0fa5a36-kube-api-access-p7p8b\") pod \"auth-test-model-5d3ac7-kserve-6c65579b75-thhpf\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:42.826200 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:42.826117    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:43.098284 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:43.098250    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"]
Sep 14 01:55:43.099662 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:55:43.099638    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod071153ee_6f09_4596_944e_08d1b0fa5a36.slice/crio-047dffbd913258ae0f13f044f75b15b7e620b4073e361c24223f00785dd82ac4 WatchSource:0}: Error finding container 047dffbd913258ae0f13f044f75b15b7e620b4073e361c24223f00785dd82ac4: Status 404 returned error can't find the container with id 047dffbd913258ae0f13f044f75b15b7e620b4073e361c24223f00785dd82ac4
Sep 14 01:55:43.832431 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:43.832394    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf" event={"ID":"071153ee-6f09-4596-944e-08d1b0fa5a36","Type":"ContainerStarted","Data":"786b556586c44c819e58407527e00bc4c1e7e8396593b021403714171aa5903f"}
Sep 14 01:55:43.832431 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:43.832434    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf" event={"ID":"071153ee-6f09-4596-944e-08d1b0fa5a36","Type":"ContainerStarted","Data":"047dffbd913258ae0f13f044f75b15b7e620b4073e361c24223f00785dd82ac4"}
Sep 14 01:55:43.848041 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:43.847994    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf" podStartSLOduration=1.847978189 podStartE2EDuration="1.847978189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:55:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:55:43.846705363 +0000 UTC m=+1503.327156184" watchObservedRunningTime="2026-09-14 01:55:43.847978189 +0000 UTC m=+1503.328429010"
Sep 14 01:55:45.578084 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:45.578038    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"]
Sep 14 01:55:50.672226 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:50.672185    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:55:52.826304 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:52.826266    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:52.826720 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:52.826329    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:52.839559 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:52.839530    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:55:52.900052 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:55:52.900023    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 01:56:00.603625 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.603574    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6" podUID="437af35c-88a2-416f-9a35-669491353849" containerName="main" containerID="cri-o://2e713f37df264f5465184533cfa088fbf021683fefc16b9857d76bd21fdc1c46" gracePeriod=45
Sep 14 01:56:00.856812 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.856744    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:56:00.934810 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.934775    2581 generic.go:356] "Generic (PLEG): container finished" podID="437af35c-88a2-416f-9a35-669491353849" containerID="2e713f37df264f5465184533cfa088fbf021683fefc16b9857d76bd21fdc1c46" exitCode=0
Sep 14 01:56:00.935018 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.934820    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"
Sep 14 01:56:00.935018 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.934870    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6" event={"ID":"437af35c-88a2-416f-9a35-669491353849","Type":"ContainerDied","Data":"2e713f37df264f5465184533cfa088fbf021683fefc16b9857d76bd21fdc1c46"}
Sep 14 01:56:00.935018 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.934925    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6" event={"ID":"437af35c-88a2-416f-9a35-669491353849","Type":"ContainerDied","Data":"354287547ec4e00a2b590441882cf0e13d5594d6a9929601a13c36dbfec8c06b"}
Sep 14 01:56:00.935018 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.934947    2581 scope.go:117] "RemoveContainer" containerID="2e713f37df264f5465184533cfa088fbf021683fefc16b9857d76bd21fdc1c46"
Sep 14 01:56:00.944200 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.944180    2581 scope.go:117] "RemoveContainer" containerID="2e713f37df264f5465184533cfa088fbf021683fefc16b9857d76bd21fdc1c46"
Sep 14 01:56:00.944500 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:56:00.944481    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e713f37df264f5465184533cfa088fbf021683fefc16b9857d76bd21fdc1c46\": container with ID starting with 2e713f37df264f5465184533cfa088fbf021683fefc16b9857d76bd21fdc1c46 not found: ID does not exist" containerID="2e713f37df264f5465184533cfa088fbf021683fefc16b9857d76bd21fdc1c46"
Sep 14 01:56:00.944546 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.944510    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e713f37df264f5465184533cfa088fbf021683fefc16b9857d76bd21fdc1c46"} err="failed to get container status \"2e713f37df264f5465184533cfa088fbf021683fefc16b9857d76bd21fdc1c46\": rpc error: code = NotFound desc = could not find container \"2e713f37df264f5465184533cfa088fbf021683fefc16b9857d76bd21fdc1c46\": container with ID starting with 2e713f37df264f5465184533cfa088fbf021683fefc16b9857d76bd21fdc1c46 not found: ID does not exist"
Sep 14 01:56:00.991670 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.991639    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/437af35c-88a2-416f-9a35-669491353849-tls-certs\") pod \"437af35c-88a2-416f-9a35-669491353849\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") "
Sep 14 01:56:00.991670 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.991672    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-home\") pod \"437af35c-88a2-416f-9a35-669491353849\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") "
Sep 14 01:56:00.991903 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.991697    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-dshm\") pod \"437af35c-88a2-416f-9a35-669491353849\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") "
Sep 14 01:56:00.991903 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.991731    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-tmp-dir\") pod \"437af35c-88a2-416f-9a35-669491353849\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") "
Sep 14 01:56:00.991903 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.991759    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-model-cache\") pod \"437af35c-88a2-416f-9a35-669491353849\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") "
Sep 14 01:56:00.991903 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.991845    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbt9l\" (UniqueName: \"kubernetes.io/projected/437af35c-88a2-416f-9a35-669491353849-kube-api-access-tbt9l\") pod \"437af35c-88a2-416f-9a35-669491353849\" (UID: \"437af35c-88a2-416f-9a35-669491353849\") "
Sep 14 01:56:00.992163 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.991987    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-home" (OuterVolumeSpecName: "home") pod "437af35c-88a2-416f-9a35-669491353849" (UID: "437af35c-88a2-416f-9a35-669491353849"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:56:00.992163 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.992087    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "437af35c-88a2-416f-9a35-669491353849" (UID: "437af35c-88a2-416f-9a35-669491353849"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:56:00.992163 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.992125    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-model-cache" (OuterVolumeSpecName: "model-cache") pod "437af35c-88a2-416f-9a35-669491353849" (UID: "437af35c-88a2-416f-9a35-669491353849"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:56:00.992313 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.992293    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:56:00.992313 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.992308    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:56:00.992431 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.992318    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:56:00.994096 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.994066    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-dshm" (OuterVolumeSpecName: "dshm") pod "437af35c-88a2-416f-9a35-669491353849" (UID: "437af35c-88a2-416f-9a35-669491353849"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:56:00.994197 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.994146    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/437af35c-88a2-416f-9a35-669491353849-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "437af35c-88a2-416f-9a35-669491353849" (UID: "437af35c-88a2-416f-9a35-669491353849"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:56:00.994197 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:00.994149    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/437af35c-88a2-416f-9a35-669491353849-kube-api-access-tbt9l" (OuterVolumeSpecName: "kube-api-access-tbt9l") pod "437af35c-88a2-416f-9a35-669491353849" (UID: "437af35c-88a2-416f-9a35-669491353849"). InnerVolumeSpecName "kube-api-access-tbt9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:56:01.094322 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:01.094280    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbt9l\" (UniqueName: \"kubernetes.io/projected/437af35c-88a2-416f-9a35-669491353849-kube-api-access-tbt9l\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:56:01.094322 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:01.094320    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/437af35c-88a2-416f-9a35-669491353849-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:56:01.094541 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:01.094338    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/437af35c-88a2-416f-9a35-669491353849-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:56:01.250143 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:01.250113    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"]
Sep 14 01:56:01.253533 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:01.253507    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/e2e-conflict-b-9e04456b-kserve-79877fb8dd-v89k6"]
Sep 14 01:56:03.054550 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:03.054519    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="437af35c-88a2-416f-9a35-669491353849" path="/var/lib/kubelet/pods/437af35c-88a2-416f-9a35-669491353849/volumes"
Sep 14 01:56:08.211305 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.211266    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"]
Sep 14 01:56:08.211697 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.211678    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="437af35c-88a2-416f-9a35-669491353849" containerName="main"
Sep 14 01:56:08.211697 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.211689    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="437af35c-88a2-416f-9a35-669491353849" containerName="main"
Sep 14 01:56:08.211770 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.211756    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="437af35c-88a2-416f-9a35-669491353849" containerName="main"
Sep 14 01:56:08.216254 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.216235    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.219360 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.219326    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-8e5935\"/\"distinct-w4-8e5935-kserve-self-signed-certs\""
Sep 14 01:56:08.221530 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.221505    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"]
Sep 14 01:56:08.369059 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.369016    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-home\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.369059 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.369059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwsf9\" (UniqueName: \"kubernetes.io/projected/b2fa3826-e86d-452c-b50c-3690f9784fda-kube-api-access-qwsf9\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.369336 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.369083    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-tmp-dir\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.369336 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.369143    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2fa3826-e86d-452c-b50c-3690f9784fda-tls-certs\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.369336 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.369160    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-dshm\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.369336 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.369236    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-model-cache\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.470527 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.470424    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-dshm\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.470527 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.470487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-model-cache\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.470773 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.470588    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-home\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.470773 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.470613    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwsf9\" (UniqueName: \"kubernetes.io/projected/b2fa3826-e86d-452c-b50c-3690f9784fda-kube-api-access-qwsf9\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.470773 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.470644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-tmp-dir\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.470773 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.470718    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2fa3826-e86d-452c-b50c-3690f9784fda-tls-certs\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.471030 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.470960    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-model-cache\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.471090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.471047    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-tmp-dir\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.471090 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.471050    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-home\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.472806 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.472785    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-dshm\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.473111 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.473094    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2fa3826-e86d-452c-b50c-3690f9784fda-tls-certs\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.477945 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.477904    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwsf9\" (UniqueName: \"kubernetes.io/projected/b2fa3826-e86d-452c-b50c-3690f9784fda-kube-api-access-qwsf9\") pod \"distinct-w4-8e5935-kserve-79ddd6c54-jf2m9\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") " pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.527146 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.527111    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:08.800084 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.800053    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"]
Sep 14 01:56:08.801188 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:56:08.801158    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2fa3826_e86d_452c_b50c_3690f9784fda.slice/crio-d9e4a71a327de4fdb84035a4c21c2c911e00487e8fc193f68e4235b108c3072d WatchSource:0}: Error finding container d9e4a71a327de4fdb84035a4c21c2c911e00487e8fc193f68e4235b108c3072d: Status 404 returned error can't find the container with id d9e4a71a327de4fdb84035a4c21c2c911e00487e8fc193f68e4235b108c3072d
Sep 14 01:56:08.985213 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.985175    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9" event={"ID":"b2fa3826-e86d-452c-b50c-3690f9784fda","Type":"ContainerStarted","Data":"b9ce0d2d10519c6c2f5d51dd8862670a201ee63575d46d21330145dfd3320144"}
Sep 14 01:56:08.985387 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:08.985225    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9" event={"ID":"b2fa3826-e86d-452c-b50c-3690f9784fda","Type":"ContainerStarted","Data":"d9e4a71a327de4fdb84035a4c21c2c911e00487e8fc193f68e4235b108c3072d"}
Sep 14 01:56:09.002079 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:09.002028    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9" podStartSLOduration=1.002012882 podStartE2EDuration="1.002012882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:56:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:56:08.998425626 +0000 UTC m=+1528.478876447" watchObservedRunningTime="2026-09-14 01:56:09.002012882 +0000 UTC m=+1528.482463702"
Sep 14 01:56:15.606106 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.606067    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"]
Sep 14 01:56:15.609934 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.609887    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.612712 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.612681    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-86995c\"/\"default-dockercfg-5gscb\""
Sep 14 01:56:15.612712 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.612690    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-86995c\"/\"kube-root-ca.crt\""
Sep 14 01:56:15.612946 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.612718    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-86995c\"/\"auth-test-model-86995c-kserve-self-signed-certs\""
Sep 14 01:56:15.612946 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.612754    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-86995c\"/\"openshift-service-ca.crt\""
Sep 14 01:56:15.619488 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.619469    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"]
Sep 14 01:56:15.749091 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.749063    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-tmp-dir\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.749271 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.749133    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-dshm\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.749271 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.749168    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzd6z\" (UniqueName: \"kubernetes.io/projected/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-kube-api-access-xzd6z\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.749271 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.749191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-tls-certs\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.749738 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.749262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-model-cache\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.749738 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.749329    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-home\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.850203 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.850169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzd6z\" (UniqueName: \"kubernetes.io/projected/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-kube-api-access-xzd6z\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.850203 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.850206    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-tls-certs\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.850423 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.850239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-model-cache\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.850423 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.850308    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-home\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.850423 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.850370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-tmp-dir\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.850557 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.850460    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-dshm\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.850722 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.850697    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-model-cache\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.850811 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.850740    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-home\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.850811 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.850761    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-tmp-dir\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.852676 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.852649    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-dshm\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.852794 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.852772    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-tls-certs\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.860955 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.860861    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzd6z\" (UniqueName: \"kubernetes.io/projected/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-kube-api-access-xzd6z\") pod \"auth-test-model-86995c-kserve-6874c8f859-c8dp4\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") " pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:15.922004 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:15.921970    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:16.195379 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:16.195348    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"]
Sep 14 01:56:16.196440 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:56:16.196411    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f08c4b4_fb46_432b_b415_f5ad1b29b5bb.slice/crio-faa4fb9b500fee8fb2b50243c472729f1f949520e86330866437538b56f6ad45 WatchSource:0}: Error finding container faa4fb9b500fee8fb2b50243c472729f1f949520e86330866437538b56f6ad45: Status 404 returned error can't find the container with id faa4fb9b500fee8fb2b50243c472729f1f949520e86330866437538b56f6ad45
Sep 14 01:56:17.034746 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:17.034708    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4" event={"ID":"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb","Type":"ContainerStarted","Data":"8372733be718afd8fcba2f46a1a087a4d93f1ff5ea6cbc89a527b6470c27285a"}
Sep 14 01:56:17.035139 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:17.034757    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4" event={"ID":"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb","Type":"ContainerStarted","Data":"faa4fb9b500fee8fb2b50243c472729f1f949520e86330866437538b56f6ad45"}
Sep 14 01:56:17.051657 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:17.051608    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4" podStartSLOduration=2.051592541 podStartE2EDuration="2.051592541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:56:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:56:17.048782494 +0000 UTC m=+1536.529233314" watchObservedRunningTime="2026-09-14 01:56:17.051592541 +0000 UTC m=+1536.532043361"
Sep 14 01:56:18.528024 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:18.527992    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:18.528438 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:18.528046    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:18.540778 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:18.540752    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:19.059036 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:19.059009    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 01:56:22.563079 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:22.563040    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:56:25.922117 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:25.922057    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:25.922575 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:25.922165    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:25.935098 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:25.935073    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:26.100964 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:26.100928    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 01:56:53.662398 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:56:53.662354    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:57:25.563233 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:57:25.563190    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:58:03.527136 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:03.527048    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"]
Sep 14 01:58:03.531287 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:03.531265    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:03.533104 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:03.533085    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-318a3c-serving-cert\""
Sep 14 01:58:03.539133 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:03.539045    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"]
Sep 14 01:58:03.593300 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:03.593262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-openshift-service-ca\") pod \"maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:03.593490 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:03.593313    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-maas-api-tls\") pod \"maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:03.593490 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:03.593349    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6szmx\" (UniqueName: \"kubernetes.io/projected/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-kube-api-access-6szmx\") pod \"maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:03.593490 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:03.593400    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-metrics-certs\") pod \"maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:03.694560 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:03.694526    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-openshift-service-ca\") pod \"maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:03.694560 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:03.694562    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-maas-api-tls\") pod \"maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:03.694837 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:03.694585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6szmx\" (UniqueName: \"kubernetes.io/projected/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-kube-api-access-6szmx\") pod \"maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:03.694837 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:03.694608    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-metrics-certs\") pod \"maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:03.694837 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:58:03.694683    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-serving-cert: secret "maas-api-e2e-api-b-318a3c-serving-cert" not found
Sep 14 01:58:03.694837 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:58:03.694765    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-maas-api-tls podName:8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb nodeName:}" failed. No retries permitted until 2026-09-14 01:58:04.194746593 +0000 UTC m=+1643.675197393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-maas-api-tls") pod "maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22" (UID: "8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb") : secret "maas-api-e2e-api-b-318a3c-serving-cert" not found
Sep 14 01:58:03.695342 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:03.695316    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-openshift-service-ca\") pod \"maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:03.697313 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:03.697281    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-metrics-certs\") pod \"maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:03.702944 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:03.702903    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6szmx\" (UniqueName: \"kubernetes.io/projected/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-kube-api-access-6szmx\") pod \"maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:04.200312 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:04.200274    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-maas-api-tls\") pod \"maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:04.202761 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:04.202742    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-maas-api-tls\") pod \"maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:04.446440 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:04.446406    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:04.722423 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:04.722392    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"]
Sep 14 01:58:04.723946 ip-10-0-132-11 kubenswrapper[2581]: W0914 01:58:04.723898    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f8d363b_ae5c_4ab2_9b57_ff9cc027c2bb.slice/crio-45c3ceae6e3f8cbe525d60203c8dc848623cb6509ea6d9ab9070fd2a9c0861a6 WatchSource:0}: Error finding container 45c3ceae6e3f8cbe525d60203c8dc848623cb6509ea6d9ab9070fd2a9c0861a6: Status 404 returned error can't find the container with id 45c3ceae6e3f8cbe525d60203c8dc848623cb6509ea6d9ab9070fd2a9c0861a6
Sep 14 01:58:04.725386 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:04.725367    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 01:58:05.705306 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:05.705263    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22" event={"ID":"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb","Type":"ContainerStarted","Data":"ee4b61cb8878d598519d8d3a044613c2aaba58b9195dd8d380b8a8daac76a5e5"}
Sep 14 01:58:05.705492 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:05.705319    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22" event={"ID":"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb","Type":"ContainerStarted","Data":"45c3ceae6e3f8cbe525d60203c8dc848623cb6509ea6d9ab9070fd2a9c0861a6"}
Sep 14 01:58:05.705492 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:05.705423    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:05.721583 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:05.721525    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22" podStartSLOduration=2.294525977 podStartE2EDuration="2.721506004s" totalImagesPullingTime="426.980027ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 01:58:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 01:58:05.719303057 +0000 UTC m=+1645.199753879" watchObservedRunningTime="2026-09-14 01:58:05.721506004 +0000 UTC m=+1645.201956891"
Sep 14 01:58:11.717575 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:11.717540    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:58:12.262832 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:12.262793    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:58:54.547663 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:54.547619    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"]
Sep 14 01:58:54.566360 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:54.566328    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"]
Sep 14 01:58:54.576143 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:54.576113    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"]
Sep 14 01:58:58.568470 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:58:58.568428    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:59:07.922470 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:07.922379    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"]
Sep 14 01:59:07.923053 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:07.922641    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22" podUID="8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb" containerName="maas-api" containerID="cri-o://ee4b61cb8878d598519d8d3a044613c2aaba58b9195dd8d380b8a8daac76a5e5" gracePeriod=30
Sep 14 01:59:08.085517 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:08.085482    2581 generic.go:356] "Generic (PLEG): container finished" podID="8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb" containerID="ee4b61cb8878d598519d8d3a044613c2aaba58b9195dd8d380b8a8daac76a5e5" exitCode=0
Sep 14 01:59:08.085720 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:08.085554    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22" event={"ID":"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb","Type":"ContainerDied","Data":"ee4b61cb8878d598519d8d3a044613c2aaba58b9195dd8d380b8a8daac76a5e5"}
Sep 14 01:59:08.180716 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:08.180648    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:59:08.276242 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:08.276187    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-maas-api-tls\") pod \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") "
Sep 14 01:59:08.276432 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:08.276367    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6szmx\" (UniqueName: \"kubernetes.io/projected/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-kube-api-access-6szmx\") pod \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") "
Sep 14 01:59:08.276432 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:08.276421    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-openshift-service-ca\") pod \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") "
Sep 14 01:59:08.276579 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:08.276446    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-metrics-certs\") pod \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\" (UID: \"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb\") "
Sep 14 01:59:08.277083 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:08.277039    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb" (UID: "8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:59:08.279099 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:08.279071    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb" (UID: "8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:59:08.279527 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:08.279502    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-kube-api-access-6szmx" (OuterVolumeSpecName: "kube-api-access-6szmx") pod "8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb" (UID: "8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb"). InnerVolumeSpecName "kube-api-access-6szmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:59:08.279635 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:08.279504    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb" (UID: "8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:59:08.377761 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:08.377715    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:08.377761 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:08.377754    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:08.377761 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:08.377764    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:08.377761 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:08.377773    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6szmx\" (UniqueName: \"kubernetes.io/projected/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb-kube-api-access-6szmx\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:09.093071 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.093037    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"
Sep 14 01:59:09.093503 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.093093    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22" event={"ID":"8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb","Type":"ContainerDied","Data":"45c3ceae6e3f8cbe525d60203c8dc848623cb6509ea6d9ab9070fd2a9c0861a6"}
Sep 14 01:59:09.093503 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.093153    2581 scope.go:117] "RemoveContainer" containerID="ee4b61cb8878d598519d8d3a044613c2aaba58b9195dd8d380b8a8daac76a5e5"
Sep 14 01:59:09.112932 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.109640    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"]
Sep 14 01:59:09.119888 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.119856    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-318a3c-5d8fffb77c-2gz22"]
Sep 14 01:59:09.568427 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.568349    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt" podUID="74e30abc-2964-475a-a83d-33cef503d42b" containerName="main" containerID="cri-o://7b4d8badff54c0666beeb30cc0224ba1246945d49673b580913e5dfb7892a306" gracePeriod=45
Sep 14 01:59:09.586355 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.586321    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68" podUID="163fadc1-20b0-4343-9f44-fc45d3730dec" containerName="main" containerID="cri-o://805c36780cab8504ba6f24165d208234ff79552f9127d7ae7b50bd3b6a094c4c" gracePeriod=45
Sep 14 01:59:09.595718 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.595684    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl" podUID="8db41348-9d3a-46e4-bc62-58bb3ffa4bdc" containerName="main" containerID="cri-o://8c2ab7f2c696379d193c2e486473b4e110d099431a755ee496c470b278a21a2b" gracePeriod=45
Sep 14 01:59:09.897851 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.897823    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:59:09.901786 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.901760    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:59:09.906132 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.906103    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:59:09.993386 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993349    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-tmp-dir\") pod \"74e30abc-2964-475a-a83d-33cef503d42b\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") "
Sep 14 01:59:09.993597 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993417    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-tmp-dir\") pod \"163fadc1-20b0-4343-9f44-fc45d3730dec\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") "
Sep 14 01:59:09.993597 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993458    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-tmp-dir\") pod \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") "
Sep 14 01:59:09.993597 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993489    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-model-cache\") pod \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") "
Sep 14 01:59:09.993597 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993538    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-tls-certs\") pod \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") "
Sep 14 01:59:09.993597 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993561    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-model-cache\") pod \"74e30abc-2964-475a-a83d-33cef503d42b\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") "
Sep 14 01:59:09.993857 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993678    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "74e30abc-2964-475a-a83d-33cef503d42b" (UID: "74e30abc-2964-475a-a83d-33cef503d42b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:59:09.993857 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993698    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8db41348-9d3a-46e4-bc62-58bb3ffa4bdc" (UID: "8db41348-9d3a-46e4-bc62-58bb3ffa4bdc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:59:09.993857 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993706    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163fadc1-20b0-4343-9f44-fc45d3730dec-tls-certs\") pod \"163fadc1-20b0-4343-9f44-fc45d3730dec\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") "
Sep 14 01:59:09.993857 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993751    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-home\") pod \"74e30abc-2964-475a-a83d-33cef503d42b\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") "
Sep 14 01:59:09.993857 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993788    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-dshm\") pod \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") "
Sep 14 01:59:09.993857 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993747    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "163fadc1-20b0-4343-9f44-fc45d3730dec" (UID: "163fadc1-20b0-4343-9f44-fc45d3730dec"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:59:09.993857 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993829    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-594fk\" (UniqueName: \"kubernetes.io/projected/163fadc1-20b0-4343-9f44-fc45d3730dec-kube-api-access-594fk\") pod \"163fadc1-20b0-4343-9f44-fc45d3730dec\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") "
Sep 14 01:59:09.993857 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993829    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-model-cache" (OuterVolumeSpecName: "model-cache") pod "8db41348-9d3a-46e4-bc62-58bb3ffa4bdc" (UID: "8db41348-9d3a-46e4-bc62-58bb3ffa4bdc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:59:09.994331 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993931    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-dshm\") pod \"74e30abc-2964-475a-a83d-33cef503d42b\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") "
Sep 14 01:59:09.994331 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993963    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-dshm\") pod \"163fadc1-20b0-4343-9f44-fc45d3730dec\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") "
Sep 14 01:59:09.994331 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.993991    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-home\") pod \"163fadc1-20b0-4343-9f44-fc45d3730dec\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") "
Sep 14 01:59:09.994331 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.994023    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgt9b\" (UniqueName: \"kubernetes.io/projected/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-kube-api-access-sgt9b\") pod \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") "
Sep 14 01:59:09.994331 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.994057    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74e30abc-2964-475a-a83d-33cef503d42b-tls-certs\") pod \"74e30abc-2964-475a-a83d-33cef503d42b\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") "
Sep 14 01:59:09.994331 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.994098    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cw79\" (UniqueName: \"kubernetes.io/projected/74e30abc-2964-475a-a83d-33cef503d42b-kube-api-access-6cw79\") pod \"74e30abc-2964-475a-a83d-33cef503d42b\" (UID: \"74e30abc-2964-475a-a83d-33cef503d42b\") "
Sep 14 01:59:09.994331 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.994127    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-home\") pod \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\" (UID: \"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc\") "
Sep 14 01:59:09.994331 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.994161    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-model-cache\") pod \"163fadc1-20b0-4343-9f44-fc45d3730dec\" (UID: \"163fadc1-20b0-4343-9f44-fc45d3730dec\") "
Sep 14 01:59:09.994753 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.994334    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-model-cache" (OuterVolumeSpecName: "model-cache") pod "74e30abc-2964-475a-a83d-33cef503d42b" (UID: "74e30abc-2964-475a-a83d-33cef503d42b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:59:09.994753 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.994600    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-home" (OuterVolumeSpecName: "home") pod "74e30abc-2964-475a-a83d-33cef503d42b" (UID: "74e30abc-2964-475a-a83d-33cef503d42b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:59:09.994753 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.994628    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:09.994753 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.994648    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:09.994753 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.994661    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:09.994753 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.994674    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:09.994753 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.994687    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:09.995142 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.994796    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-home" (OuterVolumeSpecName: "home") pod "163fadc1-20b0-4343-9f44-fc45d3730dec" (UID: "163fadc1-20b0-4343-9f44-fc45d3730dec"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:59:09.996679 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.996645    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/163fadc1-20b0-4343-9f44-fc45d3730dec-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "163fadc1-20b0-4343-9f44-fc45d3730dec" (UID: "163fadc1-20b0-4343-9f44-fc45d3730dec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:59:09.996825 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.996701    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-dshm" (OuterVolumeSpecName: "dshm") pod "8db41348-9d3a-46e4-bc62-58bb3ffa4bdc" (UID: "8db41348-9d3a-46e4-bc62-58bb3ffa4bdc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:59:09.996897 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.996830    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74e30abc-2964-475a-a83d-33cef503d42b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "74e30abc-2964-475a-a83d-33cef503d42b" (UID: "74e30abc-2964-475a-a83d-33cef503d42b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:59:09.997046 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.996991    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8db41348-9d3a-46e4-bc62-58bb3ffa4bdc" (UID: "8db41348-9d3a-46e4-bc62-58bb3ffa4bdc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:59:09.997126 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.997096    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-model-cache" (OuterVolumeSpecName: "model-cache") pod "163fadc1-20b0-4343-9f44-fc45d3730dec" (UID: "163fadc1-20b0-4343-9f44-fc45d3730dec"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:59:09.997126 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.997110    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-home" (OuterVolumeSpecName: "home") pod "8db41348-9d3a-46e4-bc62-58bb3ffa4bdc" (UID: "8db41348-9d3a-46e4-bc62-58bb3ffa4bdc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:59:09.997405 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.997384    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74e30abc-2964-475a-a83d-33cef503d42b-kube-api-access-6cw79" (OuterVolumeSpecName: "kube-api-access-6cw79") pod "74e30abc-2964-475a-a83d-33cef503d42b" (UID: "74e30abc-2964-475a-a83d-33cef503d42b"). InnerVolumeSpecName "kube-api-access-6cw79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:59:09.998062 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.998037    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-dshm" (OuterVolumeSpecName: "dshm") pod "163fadc1-20b0-4343-9f44-fc45d3730dec" (UID: "163fadc1-20b0-4343-9f44-fc45d3730dec"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:59:09.998062 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.998047    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/163fadc1-20b0-4343-9f44-fc45d3730dec-kube-api-access-594fk" (OuterVolumeSpecName: "kube-api-access-594fk") pod "163fadc1-20b0-4343-9f44-fc45d3730dec" (UID: "163fadc1-20b0-4343-9f44-fc45d3730dec"). InnerVolumeSpecName "kube-api-access-594fk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:59:09.998387 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.998366    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-dshm" (OuterVolumeSpecName: "dshm") pod "74e30abc-2964-475a-a83d-33cef503d42b" (UID: "74e30abc-2964-475a-a83d-33cef503d42b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 01:59:09.998741 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:09.998721    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-kube-api-access-sgt9b" (OuterVolumeSpecName: "kube-api-access-sgt9b") pod "8db41348-9d3a-46e4-bc62-58bb3ffa4bdc" (UID: "8db41348-9d3a-46e4-bc62-58bb3ffa4bdc"). InnerVolumeSpecName "kube-api-access-sgt9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:59:10.095443 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.095411    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163fadc1-20b0-4343-9f44-fc45d3730dec-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:10.095443 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.095436    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:10.095443 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.095445    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:10.095443 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.095453    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-594fk\" (UniqueName: \"kubernetes.io/projected/163fadc1-20b0-4343-9f44-fc45d3730dec-kube-api-access-594fk\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:10.095932 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.095461    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74e30abc-2964-475a-a83d-33cef503d42b-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:10.095932 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.095469    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:10.095932 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.095476    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:10.095932 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.095495    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgt9b\" (UniqueName: \"kubernetes.io/projected/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-kube-api-access-sgt9b\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:10.095932 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.095503    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74e30abc-2964-475a-a83d-33cef503d42b-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:10.095932 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.095513    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cw79\" (UniqueName: \"kubernetes.io/projected/74e30abc-2964-475a-a83d-33cef503d42b-kube-api-access-6cw79\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:10.095932 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.095520    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:10.095932 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.095527    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/163fadc1-20b0-4343-9f44-fc45d3730dec-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:10.095932 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.095535    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:10.101399 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.101370    2581 generic.go:356] "Generic (PLEG): container finished" podID="74e30abc-2964-475a-a83d-33cef503d42b" containerID="7b4d8badff54c0666beeb30cc0224ba1246945d49673b580913e5dfb7892a306" exitCode=0
Sep 14 01:59:10.101573 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.101411    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"
Sep 14 01:59:10.101573 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.101454    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt" event={"ID":"74e30abc-2964-475a-a83d-33cef503d42b","Type":"ContainerDied","Data":"7b4d8badff54c0666beeb30cc0224ba1246945d49673b580913e5dfb7892a306"}
Sep 14 01:59:10.101573 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.101494    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt" event={"ID":"74e30abc-2964-475a-a83d-33cef503d42b","Type":"ContainerDied","Data":"c5499a24f3347c9b860e14bb880aed9fcd19d7a162a9221767a6fa9bd6a037ab"}
Sep 14 01:59:10.101573 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.101511    2581 scope.go:117] "RemoveContainer" containerID="7b4d8badff54c0666beeb30cc0224ba1246945d49673b580913e5dfb7892a306"
Sep 14 01:59:10.103020 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.102899    2581 generic.go:356] "Generic (PLEG): container finished" podID="8db41348-9d3a-46e4-bc62-58bb3ffa4bdc" containerID="8c2ab7f2c696379d193c2e486473b4e110d099431a755ee496c470b278a21a2b" exitCode=0
Sep 14 01:59:10.103020 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.102958    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"
Sep 14 01:59:10.103020 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.102962    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl" event={"ID":"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc","Type":"ContainerDied","Data":"8c2ab7f2c696379d193c2e486473b4e110d099431a755ee496c470b278a21a2b"}
Sep 14 01:59:10.103020 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.102999    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl" event={"ID":"8db41348-9d3a-46e4-bc62-58bb3ffa4bdc","Type":"ContainerDied","Data":"ea03cb6088ffd193f18670385ca39844657f97a621543af7798c3dfd17b534be"}
Sep 14 01:59:10.104884 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.104863    2581 generic.go:356] "Generic (PLEG): container finished" podID="163fadc1-20b0-4343-9f44-fc45d3730dec" containerID="805c36780cab8504ba6f24165d208234ff79552f9127d7ae7b50bd3b6a094c4c" exitCode=0
Sep 14 01:59:10.105043 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.104949    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"
Sep 14 01:59:10.105043 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.104950    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68" event={"ID":"163fadc1-20b0-4343-9f44-fc45d3730dec","Type":"ContainerDied","Data":"805c36780cab8504ba6f24165d208234ff79552f9127d7ae7b50bd3b6a094c4c"}
Sep 14 01:59:10.105043 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.104990    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68" event={"ID":"163fadc1-20b0-4343-9f44-fc45d3730dec","Type":"ContainerDied","Data":"17bff756da8843fd0f4fc57b0f24cc4d92596c8bb19115c850e6b556732f5d2e"}
Sep 14 01:59:10.112127 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.112103    2581 scope.go:117] "RemoveContainer" containerID="7b4d8badff54c0666beeb30cc0224ba1246945d49673b580913e5dfb7892a306"
Sep 14 01:59:10.112443 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:59:10.112421    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b4d8badff54c0666beeb30cc0224ba1246945d49673b580913e5dfb7892a306\": container with ID starting with 7b4d8badff54c0666beeb30cc0224ba1246945d49673b580913e5dfb7892a306 not found: ID does not exist" containerID="7b4d8badff54c0666beeb30cc0224ba1246945d49673b580913e5dfb7892a306"
Sep 14 01:59:10.112524 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.112451    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b4d8badff54c0666beeb30cc0224ba1246945d49673b580913e5dfb7892a306"} err="failed to get container status \"7b4d8badff54c0666beeb30cc0224ba1246945d49673b580913e5dfb7892a306\": rpc error: code = NotFound desc = could not find container \"7b4d8badff54c0666beeb30cc0224ba1246945d49673b580913e5dfb7892a306\": container with ID starting with 7b4d8badff54c0666beeb30cc0224ba1246945d49673b580913e5dfb7892a306 not found: ID does not exist"
Sep 14 01:59:10.112524 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.112468    2581 scope.go:117] "RemoveContainer" containerID="8c2ab7f2c696379d193c2e486473b4e110d099431a755ee496c470b278a21a2b"
Sep 14 01:59:10.122018 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.121999    2581 scope.go:117] "RemoveContainer" containerID="8c2ab7f2c696379d193c2e486473b4e110d099431a755ee496c470b278a21a2b"
Sep 14 01:59:10.122370 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:59:10.122351    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c2ab7f2c696379d193c2e486473b4e110d099431a755ee496c470b278a21a2b\": container with ID starting with 8c2ab7f2c696379d193c2e486473b4e110d099431a755ee496c470b278a21a2b not found: ID does not exist" containerID="8c2ab7f2c696379d193c2e486473b4e110d099431a755ee496c470b278a21a2b"
Sep 14 01:59:10.122417 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.122380    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c2ab7f2c696379d193c2e486473b4e110d099431a755ee496c470b278a21a2b"} err="failed to get container status \"8c2ab7f2c696379d193c2e486473b4e110d099431a755ee496c470b278a21a2b\": rpc error: code = NotFound desc = could not find container \"8c2ab7f2c696379d193c2e486473b4e110d099431a755ee496c470b278a21a2b\": container with ID starting with 8c2ab7f2c696379d193c2e486473b4e110d099431a755ee496c470b278a21a2b not found: ID does not exist"
Sep 14 01:59:10.122417 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.122404    2581 scope.go:117] "RemoveContainer" containerID="805c36780cab8504ba6f24165d208234ff79552f9127d7ae7b50bd3b6a094c4c"
Sep 14 01:59:10.126493 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.126469    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"]
Sep 14 01:59:10.132539 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.132520    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-99c616/distinct-2-w0-99c616-kserve-5497c76956-pdfkt"]
Sep 14 01:59:10.132608 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.132559    2581 scope.go:117] "RemoveContainer" containerID="805c36780cab8504ba6f24165d208234ff79552f9127d7ae7b50bd3b6a094c4c"
Sep 14 01:59:10.132887 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:59:10.132869    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"805c36780cab8504ba6f24165d208234ff79552f9127d7ae7b50bd3b6a094c4c\": container with ID starting with 805c36780cab8504ba6f24165d208234ff79552f9127d7ae7b50bd3b6a094c4c not found: ID does not exist" containerID="805c36780cab8504ba6f24165d208234ff79552f9127d7ae7b50bd3b6a094c4c"
Sep 14 01:59:10.132972 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.132893    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"805c36780cab8504ba6f24165d208234ff79552f9127d7ae7b50bd3b6a094c4c"} err="failed to get container status \"805c36780cab8504ba6f24165d208234ff79552f9127d7ae7b50bd3b6a094c4c\": rpc error: code = NotFound desc = could not find container \"805c36780cab8504ba6f24165d208234ff79552f9127d7ae7b50bd3b6a094c4c\": container with ID starting with 805c36780cab8504ba6f24165d208234ff79552f9127d7ae7b50bd3b6a094c4c not found: ID does not exist"
Sep 14 01:59:10.144278 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.144248    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"]
Sep 14 01:59:10.147101 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.147075    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-99c616/premium-w0-99c616-kserve-748b9cbb4d-qsz68"]
Sep 14 01:59:10.154433 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.154408    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"]
Sep 14 01:59:10.157226 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:10.157201    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-99c616/simulator-w0-99c616-kserve-868ff5d65-mrrrl"]
Sep 14 01:59:11.054927 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:11.054884    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="163fadc1-20b0-4343-9f44-fc45d3730dec" path="/var/lib/kubelet/pods/163fadc1-20b0-4343-9f44-fc45d3730dec/volumes"
Sep 14 01:59:11.055371 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:11.055357    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74e30abc-2964-475a-a83d-33cef503d42b" path="/var/lib/kubelet/pods/74e30abc-2964-475a-a83d-33cef503d42b/volumes"
Sep 14 01:59:11.055722 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:11.055710    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8db41348-9d3a-46e4-bc62-58bb3ffa4bdc" path="/var/lib/kubelet/pods/8db41348-9d3a-46e4-bc62-58bb3ffa4bdc/volumes"
Sep 14 01:59:11.056477 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:11.056458    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb" path="/var/lib/kubelet/pods/8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb/volumes"
Sep 14 01:59:25.303796 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:25.303756    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"]
Sep 14 01:59:25.304389 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:25.304033    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z" podUID="5f1c351f-d602-413a-8ac3-584bc7e21146" containerName="maas-api" containerID="cri-o://e75c67410999a6d76a3cfbfde979f102fe7f9e9bd5595114d4ff5418b9e3ddea" gracePeriod=30
Sep 14 01:59:25.558427 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:25.558360    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:59:25.653755 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:25.653722    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1c351f-d602-413a-8ac3-584bc7e21146-metrics-certs\") pod \"5f1c351f-d602-413a-8ac3-584bc7e21146\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") "
Sep 14 01:59:25.653967 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:25.653769    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1c351f-d602-413a-8ac3-584bc7e21146-maas-api-tls\") pod \"5f1c351f-d602-413a-8ac3-584bc7e21146\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") "
Sep 14 01:59:25.653967 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:25.653939    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f1c351f-d602-413a-8ac3-584bc7e21146-openshift-service-ca\") pod \"5f1c351f-d602-413a-8ac3-584bc7e21146\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") "
Sep 14 01:59:25.654093 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:25.654005    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvqm9\" (UniqueName: \"kubernetes.io/projected/5f1c351f-d602-413a-8ac3-584bc7e21146-kube-api-access-gvqm9\") pod \"5f1c351f-d602-413a-8ac3-584bc7e21146\" (UID: \"5f1c351f-d602-413a-8ac3-584bc7e21146\") "
Sep 14 01:59:25.654298 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:25.654266    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f1c351f-d602-413a-8ac3-584bc7e21146-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5f1c351f-d602-413a-8ac3-584bc7e21146" (UID: "5f1c351f-d602-413a-8ac3-584bc7e21146"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:59:25.656035 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:25.656002    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f1c351f-d602-413a-8ac3-584bc7e21146-kube-api-access-gvqm9" (OuterVolumeSpecName: "kube-api-access-gvqm9") pod "5f1c351f-d602-413a-8ac3-584bc7e21146" (UID: "5f1c351f-d602-413a-8ac3-584bc7e21146"). InnerVolumeSpecName "kube-api-access-gvqm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:59:25.656156 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:25.656073    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f1c351f-d602-413a-8ac3-584bc7e21146-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5f1c351f-d602-413a-8ac3-584bc7e21146" (UID: "5f1c351f-d602-413a-8ac3-584bc7e21146"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:59:25.656156 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:25.656098    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f1c351f-d602-413a-8ac3-584bc7e21146-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5f1c351f-d602-413a-8ac3-584bc7e21146" (UID: "5f1c351f-d602-413a-8ac3-584bc7e21146"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:59:25.755255 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:25.755220    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f1c351f-d602-413a-8ac3-584bc7e21146-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:25.755255 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:25.755247    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvqm9\" (UniqueName: \"kubernetes.io/projected/5f1c351f-d602-413a-8ac3-584bc7e21146-kube-api-access-gvqm9\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:25.755255 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:25.755257    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1c351f-d602-413a-8ac3-584bc7e21146-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:25.755255 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:25.755265    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f1c351f-d602-413a-8ac3-584bc7e21146-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:26.207023 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:26.206987    2581 generic.go:356] "Generic (PLEG): container finished" podID="5f1c351f-d602-413a-8ac3-584bc7e21146" containerID="e75c67410999a6d76a3cfbfde979f102fe7f9e9bd5595114d4ff5418b9e3ddea" exitCode=0
Sep 14 01:59:26.207206 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:26.207044    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"
Sep 14 01:59:26.207206 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:26.207074    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z" event={"ID":"5f1c351f-d602-413a-8ac3-584bc7e21146","Type":"ContainerDied","Data":"e75c67410999a6d76a3cfbfde979f102fe7f9e9bd5595114d4ff5418b9e3ddea"}
Sep 14 01:59:26.207206 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:26.207156    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z" event={"ID":"5f1c351f-d602-413a-8ac3-584bc7e21146","Type":"ContainerDied","Data":"1b38eea6ef2b89a4dcc335846a491bc0d43b3c8aed02279f7bea14b6a39ca29c"}
Sep 14 01:59:26.207206 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:26.207180    2581 scope.go:117] "RemoveContainer" containerID="e75c67410999a6d76a3cfbfde979f102fe7f9e9bd5595114d4ff5418b9e3ddea"
Sep 14 01:59:26.217704 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:26.217686    2581 scope.go:117] "RemoveContainer" containerID="e75c67410999a6d76a3cfbfde979f102fe7f9e9bd5595114d4ff5418b9e3ddea"
Sep 14 01:59:26.218010 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:59:26.217988    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e75c67410999a6d76a3cfbfde979f102fe7f9e9bd5595114d4ff5418b9e3ddea\": container with ID starting with e75c67410999a6d76a3cfbfde979f102fe7f9e9bd5595114d4ff5418b9e3ddea not found: ID does not exist" containerID="e75c67410999a6d76a3cfbfde979f102fe7f9e9bd5595114d4ff5418b9e3ddea"
Sep 14 01:59:26.218077 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:26.218018    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e75c67410999a6d76a3cfbfde979f102fe7f9e9bd5595114d4ff5418b9e3ddea"} err="failed to get container status \"e75c67410999a6d76a3cfbfde979f102fe7f9e9bd5595114d4ff5418b9e3ddea\": rpc error: code = NotFound desc = could not find container \"e75c67410999a6d76a3cfbfde979f102fe7f9e9bd5595114d4ff5418b9e3ddea\": container with ID starting with e75c67410999a6d76a3cfbfde979f102fe7f9e9bd5595114d4ff5418b9e3ddea not found: ID does not exist"
Sep 14 01:59:26.228970 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:26.228943    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"]
Sep 14 01:59:26.233107 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:26.233082    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-38e192-6c648bdf8d-bcl6z"]
Sep 14 01:59:27.054522 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:27.054488    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f1c351f-d602-413a-8ac3-584bc7e21146" path="/var/lib/kubelet/pods/5f1c351f-d602-413a-8ac3-584bc7e21146/volumes"
Sep 14 01:59:39.260702 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:39.260667    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 01:59:43.325764 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.325731    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"]
Sep 14 01:59:43.326241 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.325986    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf" podUID="e9e6352f-059f-4a13-b214-52799a9fbc55" containerName="maas-api" containerID="cri-o://9d3785d577a8380dc2e8b8b0697cee0f36a907db36b3b864cc1aaf3e98a4f150" gracePeriod=30
Sep 14 01:59:43.409797 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.409758    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf" podUID="e9e6352f-059f-4a13-b214-52799a9fbc55" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.68:8443/health\": dial tcp 10.132.0.68:8443: connect: connection refused"
Sep 14 01:59:43.579567 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.579495    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:59:43.628670 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.628634    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-metrics-certs\") pod \"e9e6352f-059f-4a13-b214-52799a9fbc55\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") "
Sep 14 01:59:43.628873 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.628713    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgtvd\" (UniqueName: \"kubernetes.io/projected/e9e6352f-059f-4a13-b214-52799a9fbc55-kube-api-access-mgtvd\") pod \"e9e6352f-059f-4a13-b214-52799a9fbc55\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") "
Sep 14 01:59:43.628873 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.628762    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-maas-api-tls\") pod \"e9e6352f-059f-4a13-b214-52799a9fbc55\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") "
Sep 14 01:59:43.628873 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.628797    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9e6352f-059f-4a13-b214-52799a9fbc55-openshift-service-ca\") pod \"e9e6352f-059f-4a13-b214-52799a9fbc55\" (UID: \"e9e6352f-059f-4a13-b214-52799a9fbc55\") "
Sep 14 01:59:43.629232 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.629203    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9e6352f-059f-4a13-b214-52799a9fbc55-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e9e6352f-059f-4a13-b214-52799a9fbc55" (UID: "e9e6352f-059f-4a13-b214-52799a9fbc55"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 01:59:43.630864 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.630836    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e9e6352f-059f-4a13-b214-52799a9fbc55" (UID: "e9e6352f-059f-4a13-b214-52799a9fbc55"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:59:43.631362 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.631341    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9e6352f-059f-4a13-b214-52799a9fbc55-kube-api-access-mgtvd" (OuterVolumeSpecName: "kube-api-access-mgtvd") pod "e9e6352f-059f-4a13-b214-52799a9fbc55" (UID: "e9e6352f-059f-4a13-b214-52799a9fbc55"). InnerVolumeSpecName "kube-api-access-mgtvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 01:59:43.631362 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.631354    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e9e6352f-059f-4a13-b214-52799a9fbc55" (UID: "e9e6352f-059f-4a13-b214-52799a9fbc55"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 01:59:43.729953 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.729900    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:43.729953 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.729945    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgtvd\" (UniqueName: \"kubernetes.io/projected/e9e6352f-059f-4a13-b214-52799a9fbc55-kube-api-access-mgtvd\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:43.729953 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.729958    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9e6352f-059f-4a13-b214-52799a9fbc55-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:43.730200 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:43.729971    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9e6352f-059f-4a13-b214-52799a9fbc55-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 01:59:44.324798 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:44.324762    2581 generic.go:356] "Generic (PLEG): container finished" podID="e9e6352f-059f-4a13-b214-52799a9fbc55" containerID="9d3785d577a8380dc2e8b8b0697cee0f36a907db36b3b864cc1aaf3e98a4f150" exitCode=0
Sep 14 01:59:44.325031 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:44.324807    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"
Sep 14 01:59:44.325031 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:44.324839    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf" event={"ID":"e9e6352f-059f-4a13-b214-52799a9fbc55","Type":"ContainerDied","Data":"9d3785d577a8380dc2e8b8b0697cee0f36a907db36b3b864cc1aaf3e98a4f150"}
Sep 14 01:59:44.325031 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:44.324897    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf" event={"ID":"e9e6352f-059f-4a13-b214-52799a9fbc55","Type":"ContainerDied","Data":"71d3bdae8f1ca07f426fa23ce68b1c2398501a76d3ee857e4989058a4ce469da"}
Sep 14 01:59:44.325031 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:44.324950    2581 scope.go:117] "RemoveContainer" containerID="9d3785d577a8380dc2e8b8b0697cee0f36a907db36b3b864cc1aaf3e98a4f150"
Sep 14 01:59:44.335648 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:44.335499    2581 scope.go:117] "RemoveContainer" containerID="9d3785d577a8380dc2e8b8b0697cee0f36a907db36b3b864cc1aaf3e98a4f150"
Sep 14 01:59:44.335875 ip-10-0-132-11 kubenswrapper[2581]: E0914 01:59:44.335792    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d3785d577a8380dc2e8b8b0697cee0f36a907db36b3b864cc1aaf3e98a4f150\": container with ID starting with 9d3785d577a8380dc2e8b8b0697cee0f36a907db36b3b864cc1aaf3e98a4f150 not found: ID does not exist" containerID="9d3785d577a8380dc2e8b8b0697cee0f36a907db36b3b864cc1aaf3e98a4f150"
Sep 14 01:59:44.335875 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:44.335814    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d3785d577a8380dc2e8b8b0697cee0f36a907db36b3b864cc1aaf3e98a4f150"} err="failed to get container status \"9d3785d577a8380dc2e8b8b0697cee0f36a907db36b3b864cc1aaf3e98a4f150\": rpc error: code = NotFound desc = could not find container \"9d3785d577a8380dc2e8b8b0697cee0f36a907db36b3b864cc1aaf3e98a4f150\": container with ID starting with 9d3785d577a8380dc2e8b8b0697cee0f36a907db36b3b864cc1aaf3e98a4f150 not found: ID does not exist"
Sep 14 01:59:44.346172 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:44.346106    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"]
Sep 14 01:59:44.349328 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:44.349303    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7ff93e-785d96595c-8s7rf"]
Sep 14 01:59:45.054929 ip-10-0-132-11 kubenswrapper[2581]: I0914 01:59:45.054876    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9e6352f-059f-4a13-b214-52799a9fbc55" path="/var/lib/kubelet/pods/e9e6352f-059f-4a13-b214-52799a9fbc55/volumes"
Sep 14 02:00:00.135984 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.135949    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d3ac7-29822520-45ltd"]
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136336    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8db41348-9d3a-46e4-bc62-58bb3ffa4bdc" containerName="main"
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136348    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8db41348-9d3a-46e4-bc62-58bb3ffa4bdc" containerName="main"
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136363    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74e30abc-2964-475a-a83d-33cef503d42b" containerName="main"
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136370    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="74e30abc-2964-475a-a83d-33cef503d42b" containerName="main"
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136377    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb" containerName="maas-api"
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136382    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb" containerName="maas-api"
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136390    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="163fadc1-20b0-4343-9f44-fc45d3730dec" containerName="main"
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136403    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="163fadc1-20b0-4343-9f44-fc45d3730dec" containerName="main"
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136419    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f1c351f-d602-413a-8ac3-584bc7e21146" containerName="maas-api"
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136425    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f1c351f-d602-413a-8ac3-584bc7e21146" containerName="maas-api"
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136432    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9e6352f-059f-4a13-b214-52799a9fbc55" containerName="maas-api"
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136437    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9e6352f-059f-4a13-b214-52799a9fbc55" containerName="maas-api"
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136494    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="163fadc1-20b0-4343-9f44-fc45d3730dec" containerName="main"
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136501    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9e6352f-059f-4a13-b214-52799a9fbc55" containerName="maas-api"
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136509    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f1c351f-d602-413a-8ac3-584bc7e21146" containerName="maas-api"
Sep 14 02:00:00.136509 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136516    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="74e30abc-2964-475a-a83d-33cef503d42b" containerName="main"
Sep 14 02:00:00.137195 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136523    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8db41348-9d3a-46e4-bc62-58bb3ffa4bdc" containerName="main"
Sep 14 02:00:00.137195 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.136528    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f8d363b-ae5c-4ab2-9b57-ff9cc027c2bb" containerName="maas-api"
Sep 14 02:00:00.141398 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.141373    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d3ac7-29822520-45ltd"
Sep 14 02:00:00.143608 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.143582    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dcfmz\""
Sep 14 02:00:00.146637 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.146607    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d3ac7-29822520-45ltd"]
Sep 14 02:00:00.155730 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.155705    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d3ac7-29822520-45ltd"
Sep 14 02:00:00.427721 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.427692    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d3ac7-29822520-45ltd"]
Sep 14 02:00:00.429590 ip-10-0-132-11 kubenswrapper[2581]: W0914 02:00:00.429562    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b1ba428_224b_4137_9e5d_12d96139be4e.slice/crio-c8811a6a8955c8b54b8c214a83bb1d496b21b61f76b6dccfe4652f4cd4f61ecb WatchSource:0}: Error finding container c8811a6a8955c8b54b8c214a83bb1d496b21b61f76b6dccfe4652f4cd4f61ecb: Status 404 returned error can't find the container with id c8811a6a8955c8b54b8c214a83bb1d496b21b61f76b6dccfe4652f4cd4f61ecb
Sep 14 02:00:00.453975 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.453945    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"]
Sep 14 02:00:00.460610 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.460582    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"]
Sep 14 02:00:00.472661 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:00.472162    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"]
Sep 14 02:00:01.428775 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:01.428735    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d3ac7-29822520-45ltd" event={"ID":"6b1ba428-224b-4137-9e5d-12d96139be4e","Type":"ContainerStarted","Data":"c8811a6a8955c8b54b8c214a83bb1d496b21b61f76b6dccfe4652f4cd4f61ecb"}
Sep 14 02:00:01.961404 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:01.961351    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:00:04.449401 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:04.449311    2581 generic.go:356] "Generic (PLEG): container finished" podID="6b1ba428-224b-4137-9e5d-12d96139be4e" containerID="2cef8dc740c8322f521b05e8580e6d10dd9d24f558274372c6c586275f4e3ea0" exitCode=0
Sep 14 02:00:04.449401 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:04.449374    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d3ac7-29822520-45ltd" event={"ID":"6b1ba428-224b-4137-9e5d-12d96139be4e","Type":"ContainerDied","Data":"2cef8dc740c8322f521b05e8580e6d10dd9d24f558274372c6c586275f4e3ea0"}
Sep 14 02:00:05.601618 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:05.601589    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d3ac7-29822520-45ltd"
Sep 14 02:00:06.463783 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:06.463743    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d3ac7-29822520-45ltd" event={"ID":"6b1ba428-224b-4137-9e5d-12d96139be4e","Type":"ContainerDied","Data":"c8811a6a8955c8b54b8c214a83bb1d496b21b61f76b6dccfe4652f4cd4f61ecb"}
Sep 14 02:00:06.463982 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:06.463799    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8811a6a8955c8b54b8c214a83bb1d496b21b61f76b6dccfe4652f4cd4f61ecb"
Sep 14 02:00:06.463982 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:06.463762    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d3ac7-29822520-45ltd"
Sep 14 02:00:15.474420 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.474349    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9" podUID="b2fa3826-e86d-452c-b50c-3690f9784fda" containerName="main" containerID="cri-o://b9ce0d2d10519c6c2f5d51dd8862670a201ee63575d46d21330145dfd3320144" gracePeriod=45
Sep 14 02:00:15.484422 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.484372    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5" podUID="161d40d2-885a-44f6-8541-b84681091614" containerName="main" containerID="cri-o://7b44c23b4816d3c2d090a4f912e1513f570b086c8489d483d43fad78d4bb8e80" gracePeriod=45
Sep 14 02:00:15.494492 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.494460    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8" podUID="db632892-fbe4-4609-a892-739324cf50f7" containerName="main" containerID="cri-o://3e64c52600dc9111e56c3985cae42f74852f54c42375ae50c92bb3265aad9dd3" gracePeriod=45
Sep 14 02:00:15.685259 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.685218    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8" podUID="db632892-fbe4-4609-a892-739324cf50f7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": dial tcp 10.132.0.57:8000: connect: connection refused"
Sep 14 02:00:15.795320 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.795297    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 02:00:15.799211 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.799191    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 02:00:15.803854 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.803833    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 02:00:15.855666 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.855635    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bpf9\" (UniqueName: \"kubernetes.io/projected/db632892-fbe4-4609-a892-739324cf50f7-kube-api-access-5bpf9\") pod \"db632892-fbe4-4609-a892-739324cf50f7\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") "
Sep 14 02:00:15.855844 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.855683    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-dshm\") pod \"b2fa3826-e86d-452c-b50c-3690f9784fda\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") "
Sep 14 02:00:15.855844 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.855714    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/161d40d2-885a-44f6-8541-b84681091614-tls-certs\") pod \"161d40d2-885a-44f6-8541-b84681091614\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") "
Sep 14 02:00:15.855844 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.855762    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-dshm\") pod \"db632892-fbe4-4609-a892-739324cf50f7\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") "
Sep 14 02:00:15.855844 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.855808    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2fa3826-e86d-452c-b50c-3690f9784fda-tls-certs\") pod \"b2fa3826-e86d-452c-b50c-3690f9784fda\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") "
Sep 14 02:00:15.855844 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.855837    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-model-cache\") pod \"db632892-fbe4-4609-a892-739324cf50f7\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") "
Sep 14 02:00:15.856139 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.855862    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-home\") pod \"db632892-fbe4-4609-a892-739324cf50f7\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") "
Sep 14 02:00:15.856139 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.855925    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-tmp-dir\") pod \"161d40d2-885a-44f6-8541-b84681091614\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") "
Sep 14 02:00:15.856139 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.856080    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-home\") pod \"b2fa3826-e86d-452c-b50c-3690f9784fda\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") "
Sep 14 02:00:15.856139 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.856115    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-model-cache\") pod \"b2fa3826-e86d-452c-b50c-3690f9784fda\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") "
Sep 14 02:00:15.856348 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.856154    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwsf9\" (UniqueName: \"kubernetes.io/projected/b2fa3826-e86d-452c-b50c-3690f9784fda-kube-api-access-qwsf9\") pod \"b2fa3826-e86d-452c-b50c-3690f9784fda\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") "
Sep 14 02:00:15.856348 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.856183    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-model-cache\") pod \"161d40d2-885a-44f6-8541-b84681091614\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") "
Sep 14 02:00:15.856348 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.856183    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "161d40d2-885a-44f6-8541-b84681091614" (UID: "161d40d2-885a-44f6-8541-b84681091614"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:15.856348 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.856212    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-dshm\") pod \"161d40d2-885a-44f6-8541-b84681091614\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") "
Sep 14 02:00:15.856348 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.856283    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-tmp-dir\") pod \"b2fa3826-e86d-452c-b50c-3690f9784fda\" (UID: \"b2fa3826-e86d-452c-b50c-3690f9784fda\") "
Sep 14 02:00:15.856348 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.856336    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnjr9\" (UniqueName: \"kubernetes.io/projected/161d40d2-885a-44f6-8541-b84681091614-kube-api-access-cnjr9\") pod \"161d40d2-885a-44f6-8541-b84681091614\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") "
Sep 14 02:00:15.856652 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.856360    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-tmp-dir\") pod \"db632892-fbe4-4609-a892-739324cf50f7\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") "
Sep 14 02:00:15.856652 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.856422    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-home\") pod \"161d40d2-885a-44f6-8541-b84681091614\" (UID: \"161d40d2-885a-44f6-8541-b84681091614\") "
Sep 14 02:00:15.856652 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.856466    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db632892-fbe4-4609-a892-739324cf50f7-tls-certs\") pod \"db632892-fbe4-4609-a892-739324cf50f7\" (UID: \"db632892-fbe4-4609-a892-739324cf50f7\") "
Sep 14 02:00:15.857479 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.856855    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.858369 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.856419    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-model-cache" (OuterVolumeSpecName: "model-cache") pod "db632892-fbe4-4609-a892-739324cf50f7" (UID: "db632892-fbe4-4609-a892-739324cf50f7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:15.858369 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.856827    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-home" (OuterVolumeSpecName: "home") pod "db632892-fbe4-4609-a892-739324cf50f7" (UID: "db632892-fbe4-4609-a892-739324cf50f7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:15.858575 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.857038    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "db632892-fbe4-4609-a892-739324cf50f7" (UID: "db632892-fbe4-4609-a892-739324cf50f7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:15.858575 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.857066    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b2fa3826-e86d-452c-b50c-3690f9784fda" (UID: "b2fa3826-e86d-452c-b50c-3690f9784fda"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:15.858575 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.857237    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-model-cache" (OuterVolumeSpecName: "model-cache") pod "b2fa3826-e86d-452c-b50c-3690f9784fda" (UID: "b2fa3826-e86d-452c-b50c-3690f9784fda"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:15.858575 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.857417    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-home" (OuterVolumeSpecName: "home") pod "161d40d2-885a-44f6-8541-b84681091614" (UID: "161d40d2-885a-44f6-8541-b84681091614"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:15.858575 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.858209    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-home" (OuterVolumeSpecName: "home") pod "b2fa3826-e86d-452c-b50c-3690f9784fda" (UID: "b2fa3826-e86d-452c-b50c-3690f9784fda"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:15.858857 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.858637    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-model-cache" (OuterVolumeSpecName: "model-cache") pod "161d40d2-885a-44f6-8541-b84681091614" (UID: "161d40d2-885a-44f6-8541-b84681091614"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:15.859206 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.859182    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-dshm" (OuterVolumeSpecName: "dshm") pod "db632892-fbe4-4609-a892-739324cf50f7" (UID: "db632892-fbe4-4609-a892-739324cf50f7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:15.859578 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.859553    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db632892-fbe4-4609-a892-739324cf50f7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "db632892-fbe4-4609-a892-739324cf50f7" (UID: "db632892-fbe4-4609-a892-739324cf50f7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:00:15.859927 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.859853    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-dshm" (OuterVolumeSpecName: "dshm") pod "b2fa3826-e86d-452c-b50c-3690f9784fda" (UID: "b2fa3826-e86d-452c-b50c-3690f9784fda"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:15.859927 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.859875    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db632892-fbe4-4609-a892-739324cf50f7-kube-api-access-5bpf9" (OuterVolumeSpecName: "kube-api-access-5bpf9") pod "db632892-fbe4-4609-a892-739324cf50f7" (UID: "db632892-fbe4-4609-a892-739324cf50f7"). InnerVolumeSpecName "kube-api-access-5bpf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:00:15.860477 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.860452    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/161d40d2-885a-44f6-8541-b84681091614-kube-api-access-cnjr9" (OuterVolumeSpecName: "kube-api-access-cnjr9") pod "161d40d2-885a-44f6-8541-b84681091614" (UID: "161d40d2-885a-44f6-8541-b84681091614"). InnerVolumeSpecName "kube-api-access-cnjr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:00:15.860649 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.860625    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/161d40d2-885a-44f6-8541-b84681091614-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "161d40d2-885a-44f6-8541-b84681091614" (UID: "161d40d2-885a-44f6-8541-b84681091614"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:00:15.861072 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.861027    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-dshm" (OuterVolumeSpecName: "dshm") pod "161d40d2-885a-44f6-8541-b84681091614" (UID: "161d40d2-885a-44f6-8541-b84681091614"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:15.861598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.861569    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2fa3826-e86d-452c-b50c-3690f9784fda-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b2fa3826-e86d-452c-b50c-3690f9784fda" (UID: "b2fa3826-e86d-452c-b50c-3690f9784fda"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:00:15.861598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.861578    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2fa3826-e86d-452c-b50c-3690f9784fda-kube-api-access-qwsf9" (OuterVolumeSpecName: "kube-api-access-qwsf9") pod "b2fa3826-e86d-452c-b50c-3690f9784fda" (UID: "b2fa3826-e86d-452c-b50c-3690f9784fda"). InnerVolumeSpecName "kube-api-access-qwsf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:00:15.957354 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957319    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnjr9\" (UniqueName: \"kubernetes.io/projected/161d40d2-885a-44f6-8541-b84681091614-kube-api-access-cnjr9\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957354 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957348    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957354 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957359    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957367    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db632892-fbe4-4609-a892-739324cf50f7-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957376    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bpf9\" (UniqueName: \"kubernetes.io/projected/db632892-fbe4-4609-a892-739324cf50f7-kube-api-access-5bpf9\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957384    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957393    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/161d40d2-885a-44f6-8541-b84681091614-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957401    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957410    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2fa3826-e86d-452c-b50c-3690f9784fda-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957417    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957424    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db632892-fbe4-4609-a892-739324cf50f7-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957431    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957440    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957447    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwsf9\" (UniqueName: \"kubernetes.io/projected/b2fa3826-e86d-452c-b50c-3690f9784fda-kube-api-access-qwsf9\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957455    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957463    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/161d40d2-885a-44f6-8541-b84681091614-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:15.957598 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:15.957471    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2fa3826-e86d-452c-b50c-3690f9784fda-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:16.528626 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.528594    2581 generic.go:356] "Generic (PLEG): container finished" podID="161d40d2-885a-44f6-8541-b84681091614" containerID="7b44c23b4816d3c2d090a4f912e1513f570b086c8489d483d43fad78d4bb8e80" exitCode=0
Sep 14 02:00:16.529149 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.528638    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"
Sep 14 02:00:16.529149 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.528683    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5" event={"ID":"161d40d2-885a-44f6-8541-b84681091614","Type":"ContainerDied","Data":"7b44c23b4816d3c2d090a4f912e1513f570b086c8489d483d43fad78d4bb8e80"}
Sep 14 02:00:16.529149 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.528723    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5" event={"ID":"161d40d2-885a-44f6-8541-b84681091614","Type":"ContainerDied","Data":"02c325e9ff44536c79a2b0d1b06cd8c4d10005be175ec6e45f5a7d2fb195f3a5"}
Sep 14 02:00:16.529149 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.528745    2581 scope.go:117] "RemoveContainer" containerID="7b44c23b4816d3c2d090a4f912e1513f570b086c8489d483d43fad78d4bb8e80"
Sep 14 02:00:16.530235 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.530214    2581 generic.go:356] "Generic (PLEG): container finished" podID="b2fa3826-e86d-452c-b50c-3690f9784fda" containerID="b9ce0d2d10519c6c2f5d51dd8862670a201ee63575d46d21330145dfd3320144" exitCode=0
Sep 14 02:00:16.530318 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.530264    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"
Sep 14 02:00:16.530363 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.530308    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9" event={"ID":"b2fa3826-e86d-452c-b50c-3690f9784fda","Type":"ContainerDied","Data":"b9ce0d2d10519c6c2f5d51dd8862670a201ee63575d46d21330145dfd3320144"}
Sep 14 02:00:16.530363 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.530351    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9" event={"ID":"b2fa3826-e86d-452c-b50c-3690f9784fda","Type":"ContainerDied","Data":"d9e4a71a327de4fdb84035a4c21c2c911e00487e8fc193f68e4235b108c3072d"}
Sep 14 02:00:16.531906 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.531874    2581 generic.go:356] "Generic (PLEG): container finished" podID="db632892-fbe4-4609-a892-739324cf50f7" containerID="3e64c52600dc9111e56c3985cae42f74852f54c42375ae50c92bb3265aad9dd3" exitCode=0
Sep 14 02:00:16.532004 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.531930    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"
Sep 14 02:00:16.532004 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.531950    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8" event={"ID":"db632892-fbe4-4609-a892-739324cf50f7","Type":"ContainerDied","Data":"3e64c52600dc9111e56c3985cae42f74852f54c42375ae50c92bb3265aad9dd3"}
Sep 14 02:00:16.532004 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.531982    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8" event={"ID":"db632892-fbe4-4609-a892-739324cf50f7","Type":"ContainerDied","Data":"4babc8891ea31cd2638fcc2dc04ad04f10bfefb4b46a52412c60028c8d1a9a77"}
Sep 14 02:00:16.539863 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.539843    2581 scope.go:117] "RemoveContainer" containerID="7b44c23b4816d3c2d090a4f912e1513f570b086c8489d483d43fad78d4bb8e80"
Sep 14 02:00:16.540262 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:00:16.540241    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b44c23b4816d3c2d090a4f912e1513f570b086c8489d483d43fad78d4bb8e80\": container with ID starting with 7b44c23b4816d3c2d090a4f912e1513f570b086c8489d483d43fad78d4bb8e80 not found: ID does not exist" containerID="7b44c23b4816d3c2d090a4f912e1513f570b086c8489d483d43fad78d4bb8e80"
Sep 14 02:00:16.540357 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.540273    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b44c23b4816d3c2d090a4f912e1513f570b086c8489d483d43fad78d4bb8e80"} err="failed to get container status \"7b44c23b4816d3c2d090a4f912e1513f570b086c8489d483d43fad78d4bb8e80\": rpc error: code = NotFound desc = could not find container \"7b44c23b4816d3c2d090a4f912e1513f570b086c8489d483d43fad78d4bb8e80\": container with ID starting with 7b44c23b4816d3c2d090a4f912e1513f570b086c8489d483d43fad78d4bb8e80 not found: ID does not exist"
Sep 14 02:00:16.540357 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.540298    2581 scope.go:117] "RemoveContainer" containerID="b9ce0d2d10519c6c2f5d51dd8862670a201ee63575d46d21330145dfd3320144"
Sep 14 02:00:16.549602 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.549585    2581 scope.go:117] "RemoveContainer" containerID="b9ce0d2d10519c6c2f5d51dd8862670a201ee63575d46d21330145dfd3320144"
Sep 14 02:00:16.549903 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:00:16.549886    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9ce0d2d10519c6c2f5d51dd8862670a201ee63575d46d21330145dfd3320144\": container with ID starting with b9ce0d2d10519c6c2f5d51dd8862670a201ee63575d46d21330145dfd3320144 not found: ID does not exist" containerID="b9ce0d2d10519c6c2f5d51dd8862670a201ee63575d46d21330145dfd3320144"
Sep 14 02:00:16.549979 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.549933    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9ce0d2d10519c6c2f5d51dd8862670a201ee63575d46d21330145dfd3320144"} err="failed to get container status \"b9ce0d2d10519c6c2f5d51dd8862670a201ee63575d46d21330145dfd3320144\": rpc error: code = NotFound desc = could not find container \"b9ce0d2d10519c6c2f5d51dd8862670a201ee63575d46d21330145dfd3320144\": container with ID starting with b9ce0d2d10519c6c2f5d51dd8862670a201ee63575d46d21330145dfd3320144 not found: ID does not exist"
Sep 14 02:00:16.549979 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.549952    2581 scope.go:117] "RemoveContainer" containerID="3e64c52600dc9111e56c3985cae42f74852f54c42375ae50c92bb3265aad9dd3"
Sep 14 02:00:16.564643 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.564609    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"]
Sep 14 02:00:16.566261 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.566237    2581 scope.go:117] "RemoveContainer" containerID="3e64c52600dc9111e56c3985cae42f74852f54c42375ae50c92bb3265aad9dd3"
Sep 14 02:00:16.566762 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:00:16.566733    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e64c52600dc9111e56c3985cae42f74852f54c42375ae50c92bb3265aad9dd3\": container with ID starting with 3e64c52600dc9111e56c3985cae42f74852f54c42375ae50c92bb3265aad9dd3 not found: ID does not exist" containerID="3e64c52600dc9111e56c3985cae42f74852f54c42375ae50c92bb3265aad9dd3"
Sep 14 02:00:16.566847 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.566777    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e64c52600dc9111e56c3985cae42f74852f54c42375ae50c92bb3265aad9dd3"} err="failed to get container status \"3e64c52600dc9111e56c3985cae42f74852f54c42375ae50c92bb3265aad9dd3\": rpc error: code = NotFound desc = could not find container \"3e64c52600dc9111e56c3985cae42f74852f54c42375ae50c92bb3265aad9dd3\": container with ID starting with 3e64c52600dc9111e56c3985cae42f74852f54c42375ae50c92bb3265aad9dd3 not found: ID does not exist"
Sep 14 02:00:16.570094 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.570069    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/distinct-w4-8e5935-kserve-79ddd6c54-jf2m9"]
Sep 14 02:00:16.578819 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.578792    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"]
Sep 14 02:00:16.581675 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.581654    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/premium-w4-8e5935-kserve-758bccfd47-9dcp5"]
Sep 14 02:00:16.589516 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.589495    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"]
Sep 14 02:00:16.593057 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:16.593030    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-8e5935/simulator-w4-8e5935-kserve-5fb756b74-q5ct8"]
Sep 14 02:00:17.054936 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:17.054884    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="161d40d2-885a-44f6-8541-b84681091614" path="/var/lib/kubelet/pods/161d40d2-885a-44f6-8541-b84681091614/volumes"
Sep 14 02:00:17.055354 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:17.055339    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2fa3826-e86d-452c-b50c-3690f9784fda" path="/var/lib/kubelet/pods/b2fa3826-e86d-452c-b50c-3690f9784fda/volumes"
Sep 14 02:00:17.055726 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:17.055713    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db632892-fbe4-4609-a892-739324cf50f7" path="/var/lib/kubelet/pods/db632892-fbe4-4609-a892-739324cf50f7/volumes"
Sep 14 02:00:23.021942 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:23.021890    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"]
Sep 14 02:00:24.554958 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:24.554907    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:00:24.915357 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:24.915272    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"]
Sep 14 02:00:37.755035 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:37.754940    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:00:38.041989 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.041883    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf" podUID="071153ee-6f09-4596-944e-08d1b0fa5a36" containerName="main" containerID="cri-o://786b556586c44c819e58407527e00bc4c1e7e8396593b021403714171aa5903f" gracePeriod=45
Sep 14 02:00:38.298394 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.298331    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 02:00:38.375088 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.375047    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-dshm\") pod \"071153ee-6f09-4596-944e-08d1b0fa5a36\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") "
Sep 14 02:00:38.375088 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.375098    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/071153ee-6f09-4596-944e-08d1b0fa5a36-tls-certs\") pod \"071153ee-6f09-4596-944e-08d1b0fa5a36\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") "
Sep 14 02:00:38.375326 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.375156    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-tmp-dir\") pod \"071153ee-6f09-4596-944e-08d1b0fa5a36\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") "
Sep 14 02:00:38.375326 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.375226    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7p8b\" (UniqueName: \"kubernetes.io/projected/071153ee-6f09-4596-944e-08d1b0fa5a36-kube-api-access-p7p8b\") pod \"071153ee-6f09-4596-944e-08d1b0fa5a36\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") "
Sep 14 02:00:38.375326 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.375268    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-model-cache\") pod \"071153ee-6f09-4596-944e-08d1b0fa5a36\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") "
Sep 14 02:00:38.375497 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.375459    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-home\") pod \"071153ee-6f09-4596-944e-08d1b0fa5a36\" (UID: \"071153ee-6f09-4596-944e-08d1b0fa5a36\") "
Sep 14 02:00:38.375557 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.375472    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "071153ee-6f09-4596-944e-08d1b0fa5a36" (UID: "071153ee-6f09-4596-944e-08d1b0fa5a36"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:38.375557 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.375530    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-model-cache" (OuterVolumeSpecName: "model-cache") pod "071153ee-6f09-4596-944e-08d1b0fa5a36" (UID: "071153ee-6f09-4596-944e-08d1b0fa5a36"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:38.375712 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.375687    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-home" (OuterVolumeSpecName: "home") pod "071153ee-6f09-4596-944e-08d1b0fa5a36" (UID: "071153ee-6f09-4596-944e-08d1b0fa5a36"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:38.375889 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.375868    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:38.375889 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.375885    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:38.375889 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.375894    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:38.377608 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.377576    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/071153ee-6f09-4596-944e-08d1b0fa5a36-kube-api-access-p7p8b" (OuterVolumeSpecName: "kube-api-access-p7p8b") pod "071153ee-6f09-4596-944e-08d1b0fa5a36" (UID: "071153ee-6f09-4596-944e-08d1b0fa5a36"). InnerVolumeSpecName "kube-api-access-p7p8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:00:38.377709 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.377608    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-dshm" (OuterVolumeSpecName: "dshm") pod "071153ee-6f09-4596-944e-08d1b0fa5a36" (UID: "071153ee-6f09-4596-944e-08d1b0fa5a36"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:38.377709 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.377621    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/071153ee-6f09-4596-944e-08d1b0fa5a36-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "071153ee-6f09-4596-944e-08d1b0fa5a36" (UID: "071153ee-6f09-4596-944e-08d1b0fa5a36"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:00:38.476588 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.476548    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7p8b\" (UniqueName: \"kubernetes.io/projected/071153ee-6f09-4596-944e-08d1b0fa5a36-kube-api-access-p7p8b\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:38.476588 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.476580    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/071153ee-6f09-4596-944e-08d1b0fa5a36-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:38.476588 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.476590    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/071153ee-6f09-4596-944e-08d1b0fa5a36-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:38.666430 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.666341    2581 generic.go:356] "Generic (PLEG): container finished" podID="071153ee-6f09-4596-944e-08d1b0fa5a36" containerID="786b556586c44c819e58407527e00bc4c1e7e8396593b021403714171aa5903f" exitCode=0
Sep 14 02:00:38.666430 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.666389    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"
Sep 14 02:00:38.666613 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.666431    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf" event={"ID":"071153ee-6f09-4596-944e-08d1b0fa5a36","Type":"ContainerDied","Data":"786b556586c44c819e58407527e00bc4c1e7e8396593b021403714171aa5903f"}
Sep 14 02:00:38.666613 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.666482    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf" event={"ID":"071153ee-6f09-4596-944e-08d1b0fa5a36","Type":"ContainerDied","Data":"047dffbd913258ae0f13f044f75b15b7e620b4073e361c24223f00785dd82ac4"}
Sep 14 02:00:38.666613 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.666504    2581 scope.go:117] "RemoveContainer" containerID="786b556586c44c819e58407527e00bc4c1e7e8396593b021403714171aa5903f"
Sep 14 02:00:38.676192 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.676166    2581 scope.go:117] "RemoveContainer" containerID="786b556586c44c819e58407527e00bc4c1e7e8396593b021403714171aa5903f"
Sep 14 02:00:38.676497 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:00:38.676480    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"786b556586c44c819e58407527e00bc4c1e7e8396593b021403714171aa5903f\": container with ID starting with 786b556586c44c819e58407527e00bc4c1e7e8396593b021403714171aa5903f not found: ID does not exist" containerID="786b556586c44c819e58407527e00bc4c1e7e8396593b021403714171aa5903f"
Sep 14 02:00:38.676550 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.676506    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"786b556586c44c819e58407527e00bc4c1e7e8396593b021403714171aa5903f"} err="failed to get container status \"786b556586c44c819e58407527e00bc4c1e7e8396593b021403714171aa5903f\": rpc error: code = NotFound desc = could not find container \"786b556586c44c819e58407527e00bc4c1e7e8396593b021403714171aa5903f\": container with ID starting with 786b556586c44c819e58407527e00bc4c1e7e8396593b021403714171aa5903f not found: ID does not exist"
Sep 14 02:00:38.686211 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.686182    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"]
Sep 14 02:00:38.692472 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:38.692443    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d3ac7/auth-test-model-5d3ac7-kserve-6c65579b75-thhpf"]
Sep 14 02:00:39.054551 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:39.054505    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="071153ee-6f09-4596-944e-08d1b0fa5a36" path="/var/lib/kubelet/pods/071153ee-6f09-4596-944e-08d1b0fa5a36/volumes"
Sep 14 02:00:39.935541 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:39.935503    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4" podUID="1f08c4b4-fb46-432b-b415-f5ad1b29b5bb" containerName="main" containerID="cri-o://8372733be718afd8fcba2f46a1a087a4d93f1ff5ea6cbc89a527b6470c27285a" gracePeriod=45
Sep 14 02:00:40.191184 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.191110    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 02:00:40.294191 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.294156    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-tmp-dir\") pod \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") "
Sep 14 02:00:40.294366 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.294258    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzd6z\" (UniqueName: \"kubernetes.io/projected/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-kube-api-access-xzd6z\") pod \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") "
Sep 14 02:00:40.294366 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.294298    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-tls-certs\") pod \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") "
Sep 14 02:00:40.294366 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.294331    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-home\") pod \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") "
Sep 14 02:00:40.294500 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.294365    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-model-cache\") pod \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") "
Sep 14 02:00:40.294563 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.294509    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1f08c4b4-fb46-432b-b415-f5ad1b29b5bb" (UID: "1f08c4b4-fb46-432b-b415-f5ad1b29b5bb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:40.294563 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.294534    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-dshm\") pod \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\" (UID: \"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb\") "
Sep 14 02:00:40.294662 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.294614    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-home" (OuterVolumeSpecName: "home") pod "1f08c4b4-fb46-432b-b415-f5ad1b29b5bb" (UID: "1f08c4b4-fb46-432b-b415-f5ad1b29b5bb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:40.294662 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.294626    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-model-cache" (OuterVolumeSpecName: "model-cache") pod "1f08c4b4-fb46-432b-b415-f5ad1b29b5bb" (UID: "1f08c4b4-fb46-432b-b415-f5ad1b29b5bb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:40.294846 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.294830    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:40.294903 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.294852    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:40.294903 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.294869    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:40.296714 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.296693    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-dshm" (OuterVolumeSpecName: "dshm") pod "1f08c4b4-fb46-432b-b415-f5ad1b29b5bb" (UID: "1f08c4b4-fb46-432b-b415-f5ad1b29b5bb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:00:40.296714 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.296704    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1f08c4b4-fb46-432b-b415-f5ad1b29b5bb" (UID: "1f08c4b4-fb46-432b-b415-f5ad1b29b5bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:00:40.296832 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.296719    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-kube-api-access-xzd6z" (OuterVolumeSpecName: "kube-api-access-xzd6z") pod "1f08c4b4-fb46-432b-b415-f5ad1b29b5bb" (UID: "1f08c4b4-fb46-432b-b415-f5ad1b29b5bb"). InnerVolumeSpecName "kube-api-access-xzd6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:00:40.395603 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.395552    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:40.395603 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.395598    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzd6z\" (UniqueName: \"kubernetes.io/projected/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-kube-api-access-xzd6z\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:40.395603 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.395611    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:00:40.680057 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.680022    2581 generic.go:356] "Generic (PLEG): container finished" podID="1f08c4b4-fb46-432b-b415-f5ad1b29b5bb" containerID="8372733be718afd8fcba2f46a1a087a4d93f1ff5ea6cbc89a527b6470c27285a" exitCode=0
Sep 14 02:00:40.680256 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.680066    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"
Sep 14 02:00:40.680256 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.680105    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4" event={"ID":"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb","Type":"ContainerDied","Data":"8372733be718afd8fcba2f46a1a087a4d93f1ff5ea6cbc89a527b6470c27285a"}
Sep 14 02:00:40.680256 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.680145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4" event={"ID":"1f08c4b4-fb46-432b-b415-f5ad1b29b5bb","Type":"ContainerDied","Data":"faa4fb9b500fee8fb2b50243c472729f1f949520e86330866437538b56f6ad45"}
Sep 14 02:00:40.680256 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.680164    2581 scope.go:117] "RemoveContainer" containerID="8372733be718afd8fcba2f46a1a087a4d93f1ff5ea6cbc89a527b6470c27285a"
Sep 14 02:00:40.696328 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.696306    2581 scope.go:117] "RemoveContainer" containerID="8372733be718afd8fcba2f46a1a087a4d93f1ff5ea6cbc89a527b6470c27285a"
Sep 14 02:00:40.696663 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:00:40.696643    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8372733be718afd8fcba2f46a1a087a4d93f1ff5ea6cbc89a527b6470c27285a\": container with ID starting with 8372733be718afd8fcba2f46a1a087a4d93f1ff5ea6cbc89a527b6470c27285a not found: ID does not exist" containerID="8372733be718afd8fcba2f46a1a087a4d93f1ff5ea6cbc89a527b6470c27285a"
Sep 14 02:00:40.696705 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.696672    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8372733be718afd8fcba2f46a1a087a4d93f1ff5ea6cbc89a527b6470c27285a"} err="failed to get container status \"8372733be718afd8fcba2f46a1a087a4d93f1ff5ea6cbc89a527b6470c27285a\": rpc error: code = NotFound desc = could not find container \"8372733be718afd8fcba2f46a1a087a4d93f1ff5ea6cbc89a527b6470c27285a\": container with ID starting with 8372733be718afd8fcba2f46a1a087a4d93f1ff5ea6cbc89a527b6470c27285a not found: ID does not exist"
Sep 14 02:00:40.704701 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.704674    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"]
Sep 14 02:00:40.706863 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:40.706841    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-86995c/auth-test-model-86995c-kserve-6874c8f859-c8dp4"]
Sep 14 02:00:41.055208 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:41.055178    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f08c4b4-fb46-432b-b415-f5ad1b29b5bb" path="/var/lib/kubelet/pods/1f08c4b4-fb46-432b-b415-f5ad1b29b5bb/volumes"
Sep 14 02:00:41.183242 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:41.183214    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 02:00:41.193324 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:41.193298    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 02:00:58.194356 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194315    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"]
Sep 14 02:00:58.194774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194682    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b1ba428-224b-4137-9e5d-12d96139be4e" containerName="cleanup"
Sep 14 02:00:58.194774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194692    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b1ba428-224b-4137-9e5d-12d96139be4e" containerName="cleanup"
Sep 14 02:00:58.194774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194701    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db632892-fbe4-4609-a892-739324cf50f7" containerName="main"
Sep 14 02:00:58.194774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194706    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="db632892-fbe4-4609-a892-739324cf50f7" containerName="main"
Sep 14 02:00:58.194774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194715    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="071153ee-6f09-4596-944e-08d1b0fa5a36" containerName="main"
Sep 14 02:00:58.194774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194721    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="071153ee-6f09-4596-944e-08d1b0fa5a36" containerName="main"
Sep 14 02:00:58.194774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194733    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f08c4b4-fb46-432b-b415-f5ad1b29b5bb" containerName="main"
Sep 14 02:00:58.194774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194738    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f08c4b4-fb46-432b-b415-f5ad1b29b5bb" containerName="main"
Sep 14 02:00:58.194774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194746    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2fa3826-e86d-452c-b50c-3690f9784fda" containerName="main"
Sep 14 02:00:58.194774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194751    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2fa3826-e86d-452c-b50c-3690f9784fda" containerName="main"
Sep 14 02:00:58.194774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194763    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="161d40d2-885a-44f6-8541-b84681091614" containerName="main"
Sep 14 02:00:58.194774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194768    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="161d40d2-885a-44f6-8541-b84681091614" containerName="main"
Sep 14 02:00:58.195244 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194826    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="db632892-fbe4-4609-a892-739324cf50f7" containerName="main"
Sep 14 02:00:58.195244 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194833    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b1ba428-224b-4137-9e5d-12d96139be4e" containerName="cleanup"
Sep 14 02:00:58.195244 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194840    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f08c4b4-fb46-432b-b415-f5ad1b29b5bb" containerName="main"
Sep 14 02:00:58.195244 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194846    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="161d40d2-885a-44f6-8541-b84681091614" containerName="main"
Sep 14 02:00:58.195244 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194852    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2fa3826-e86d-452c-b50c-3690f9784fda" containerName="main"
Sep 14 02:00:58.195244 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.194859    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="071153ee-6f09-4596-944e-08d1b0fa5a36" containerName="main"
Sep 14 02:00:58.198003 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.197980    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.200122 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.200089    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-86995c\"/\"openshift-service-ca.crt\""
Sep 14 02:00:58.200122 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.200115    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-86995c\"/\"kube-root-ca.crt\""
Sep 14 02:00:58.200863 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.200843    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-86995c\"/\"default-dockercfg-5gscb\""
Sep 14 02:00:58.201024 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.200843    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-86995c\"/\"sub-test-model-86995c-kserve-self-signed-certs\""
Sep 14 02:00:58.208100 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.208074    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"]
Sep 14 02:00:58.263641 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.263589    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d8b6081-a265-4cd4-91ce-0f9686345c82-tls-certs\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.263830 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.263655    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-tmp-dir\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.263830 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.263722    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-dshm\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.263830 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.263759    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-home\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.263830 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.263809    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzn5j\" (UniqueName: \"kubernetes.io/projected/8d8b6081-a265-4cd4-91ce-0f9686345c82-kube-api-access-tzn5j\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.263995 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.263834    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-model-cache\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.364879 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.364841    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-dshm\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.364879 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.364890    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-home\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.365198 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.365031    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzn5j\" (UniqueName: \"kubernetes.io/projected/8d8b6081-a265-4cd4-91ce-0f9686345c82-kube-api-access-tzn5j\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.365198 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.365067    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-model-cache\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.365198 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.365123    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d8b6081-a265-4cd4-91ce-0f9686345c82-tls-certs\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.365198 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.365143    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-tmp-dir\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.365466 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.365450    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-home\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.365553 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.365526    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-model-cache\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.365602 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.365558    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-tmp-dir\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.367378 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.367351    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-dshm\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.367630 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.367613    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d8b6081-a265-4cd4-91ce-0f9686345c82-tls-certs\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.372466 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.372442    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzn5j\" (UniqueName: \"kubernetes.io/projected/8d8b6081-a265-4cd4-91ce-0f9686345c82-kube-api-access-tzn5j\") pod \"sub-test-model-86995c-kserve-66b7f95bc9-nzhwh\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") " pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.508674 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.508633    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:00:58.780827 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:58.780803    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"]
Sep 14 02:00:58.783620 ip-10-0-132-11 kubenswrapper[2581]: W0914 02:00:58.783585    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d8b6081_a265_4cd4_91ce_0f9686345c82.slice/crio-672bfdc5a9833ae1aba8ffc5f849ac5d34cc574cd2485cab07357bf1f3bdba7f WatchSource:0}: Error finding container 672bfdc5a9833ae1aba8ffc5f849ac5d34cc574cd2485cab07357bf1f3bdba7f: Status 404 returned error can't find the container with id 672bfdc5a9833ae1aba8ffc5f849ac5d34cc574cd2485cab07357bf1f3bdba7f
Sep 14 02:00:59.786785 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:59.786749    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh" event={"ID":"8d8b6081-a265-4cd4-91ce-0f9686345c82","Type":"ContainerStarted","Data":"500aded26b9637724f8922fd51dd0864fabfdac6bd353efbcdfbbe043be389ab"}
Sep 14 02:00:59.786785 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:59.786802    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh" event={"ID":"8d8b6081-a265-4cd4-91ce-0f9686345c82","Type":"ContainerStarted","Data":"672bfdc5a9833ae1aba8ffc5f849ac5d34cc574cd2485cab07357bf1f3bdba7f"}
Sep 14 02:00:59.803590 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:00:59.803534    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh" podStartSLOduration=1.803517266 podStartE2EDuration="1.803517266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:00:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:00:59.800980669 +0000 UTC m=+1819.281431489" watchObservedRunningTime="2026-09-14 02:00:59.803517266 +0000 UTC m=+1819.283968088"
Sep 14 02:01:01.865771 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:01.865731    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:01:08.509179 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:08.509132    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:01:08.509179 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:08.509190    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:01:08.522122 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:08.522089    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:01:08.853288 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:08.853203    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:01:22.762192 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:22.762155    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:01:36.059225 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:36.059191    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:01:46.314903 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:46.314857    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"]
Sep 14 02:01:48.458335 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:48.458291    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:01:51.011835 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.011795    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"]
Sep 14 02:01:51.015333 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.015309    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.017482 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.017461    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d3ac7\"/\"openshift-service-ca.crt\""
Sep 14 02:01:51.017612 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.017466    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d3ac7\"/\"rate-test-model-5d3ac7-kserve-self-signed-certs\""
Sep 14 02:01:51.018075 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.018059    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d3ac7\"/\"kube-root-ca.crt\""
Sep 14 02:01:51.018358 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.018338    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5d3ac7\"/\"default-dockercfg-gw7rs\""
Sep 14 02:01:51.024624 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.024598    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"]
Sep 14 02:01:51.080538 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.080507    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f242c87-cfd7-4f36-a43c-17d1137186fd-tls-certs\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.080722 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.080552    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-tmp-dir\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.080722 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.080571    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-model-cache\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.080722 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.080616    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-dshm\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.080722 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.080646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg8qj\" (UniqueName: \"kubernetes.io/projected/7f242c87-cfd7-4f36-a43c-17d1137186fd-kube-api-access-mg8qj\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.080866 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.080730    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-home\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.181685 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.181643    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg8qj\" (UniqueName: \"kubernetes.io/projected/7f242c87-cfd7-4f36-a43c-17d1137186fd-kube-api-access-mg8qj\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.181880 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.181702    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-home\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.181880 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.181753    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f242c87-cfd7-4f36-a43c-17d1137186fd-tls-certs\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.181880 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.181781    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-tmp-dir\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.181880 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.181807    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-model-cache\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.181880 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.181853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-dshm\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.182185 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.182166    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-home\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.182252 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.182226    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-tmp-dir\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.182368 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.182347    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-model-cache\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.184254 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.184234    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-dshm\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.184671 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.184651    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f242c87-cfd7-4f36-a43c-17d1137186fd-tls-certs\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.188359 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.188336    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg8qj\" (UniqueName: \"kubernetes.io/projected/7f242c87-cfd7-4f36-a43c-17d1137186fd-kube-api-access-mg8qj\") pod \"rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") " pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.325871 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.325774    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:01:51.599173 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:51.599146    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"]
Sep 14 02:01:51.601136 ip-10-0-132-11 kubenswrapper[2581]: W0914 02:01:51.601098    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f242c87_cfd7_4f36_a43c_17d1137186fd.slice/crio-194e7108e0094533cecf46199c73dfab4d41f621e39331cec60715ec1c1f3aad WatchSource:0}: Error finding container 194e7108e0094533cecf46199c73dfab4d41f621e39331cec60715ec1c1f3aad: Status 404 returned error can't find the container with id 194e7108e0094533cecf46199c73dfab4d41f621e39331cec60715ec1c1f3aad
Sep 14 02:01:52.085974 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:52.085933    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc" event={"ID":"7f242c87-cfd7-4f36-a43c-17d1137186fd","Type":"ContainerStarted","Data":"9958bcf54e514001b1ee9ac6d3b6b394a25cadfbf01c8d0b9a812615d9228f22"}
Sep 14 02:01:52.085974 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:52.085982    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc" event={"ID":"7f242c87-cfd7-4f36-a43c-17d1137186fd","Type":"ContainerStarted","Data":"194e7108e0094533cecf46199c73dfab4d41f621e39331cec60715ec1c1f3aad"}
Sep 14 02:01:52.102039 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:52.101981    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc" podStartSLOduration=2.101964208 podStartE2EDuration="2.101964208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:01:52.099499039 +0000 UTC m=+1871.579949860" watchObservedRunningTime="2026-09-14 02:01:52.101964208 +0000 UTC m=+1871.582415028"
Sep 14 02:01:58.856198 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:01:58.856163    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:02:01.325977 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.325935    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:02:01.326486 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.325991    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:02:01.335702 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.335659    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh" podUID="8d8b6081-a265-4cd4-91ce-0f9686345c82" containerName="main" containerID="cri-o://500aded26b9637724f8922fd51dd0864fabfdac6bd353efbcdfbbe043be389ab" gracePeriod=45
Sep 14 02:02:01.338842 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.338818    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:02:01.598774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.598743    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:02:01.791119 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.791077    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-model-cache\") pod \"8d8b6081-a265-4cd4-91ce-0f9686345c82\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") "
Sep 14 02:02:01.791318 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.791181    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d8b6081-a265-4cd4-91ce-0f9686345c82-tls-certs\") pod \"8d8b6081-a265-4cd4-91ce-0f9686345c82\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") "
Sep 14 02:02:01.791318 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.791209    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-home\") pod \"8d8b6081-a265-4cd4-91ce-0f9686345c82\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") "
Sep 14 02:02:01.791318 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.791242    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-tmp-dir\") pod \"8d8b6081-a265-4cd4-91ce-0f9686345c82\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") "
Sep 14 02:02:01.791502 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.791372    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-model-cache" (OuterVolumeSpecName: "model-cache") pod "8d8b6081-a265-4cd4-91ce-0f9686345c82" (UID: "8d8b6081-a265-4cd4-91ce-0f9686345c82"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:02:01.791502 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.791403    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-dshm\") pod \"8d8b6081-a265-4cd4-91ce-0f9686345c82\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") "
Sep 14 02:02:01.791502 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.791461    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzn5j\" (UniqueName: \"kubernetes.io/projected/8d8b6081-a265-4cd4-91ce-0f9686345c82-kube-api-access-tzn5j\") pod \"8d8b6081-a265-4cd4-91ce-0f9686345c82\" (UID: \"8d8b6081-a265-4cd4-91ce-0f9686345c82\") "
Sep 14 02:02:01.791651 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.791499    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8d8b6081-a265-4cd4-91ce-0f9686345c82" (UID: "8d8b6081-a265-4cd4-91ce-0f9686345c82"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:02:01.791651 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.791514    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-home" (OuterVolumeSpecName: "home") pod "8d8b6081-a265-4cd4-91ce-0f9686345c82" (UID: "8d8b6081-a265-4cd4-91ce-0f9686345c82"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:02:01.791797 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.791777    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:02:01.791858 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.791802    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:02:01.791858 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.791811    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:02:01.793498 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.793474    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d8b6081-a265-4cd4-91ce-0f9686345c82-kube-api-access-tzn5j" (OuterVolumeSpecName: "kube-api-access-tzn5j") pod "8d8b6081-a265-4cd4-91ce-0f9686345c82" (UID: "8d8b6081-a265-4cd4-91ce-0f9686345c82"). InnerVolumeSpecName "kube-api-access-tzn5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:02:01.793630 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.793501    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-dshm" (OuterVolumeSpecName: "dshm") pod "8d8b6081-a265-4cd4-91ce-0f9686345c82" (UID: "8d8b6081-a265-4cd4-91ce-0f9686345c82"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:02:01.793630 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.793505    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d8b6081-a265-4cd4-91ce-0f9686345c82-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8d8b6081-a265-4cd4-91ce-0f9686345c82" (UID: "8d8b6081-a265-4cd4-91ce-0f9686345c82"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:02:01.892943 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.892818    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d8b6081-a265-4cd4-91ce-0f9686345c82-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:02:01.892943 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.892853    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d8b6081-a265-4cd4-91ce-0f9686345c82-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:02:01.892943 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:01.892863    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzn5j\" (UniqueName: \"kubernetes.io/projected/8d8b6081-a265-4cd4-91ce-0f9686345c82-kube-api-access-tzn5j\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:02:02.142177 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:02.142145    2581 generic.go:356] "Generic (PLEG): container finished" podID="8d8b6081-a265-4cd4-91ce-0f9686345c82" containerID="500aded26b9637724f8922fd51dd0864fabfdac6bd353efbcdfbbe043be389ab" exitCode=0
Sep 14 02:02:02.142372 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:02.142191    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"
Sep 14 02:02:02.142372 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:02.142232    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh" event={"ID":"8d8b6081-a265-4cd4-91ce-0f9686345c82","Type":"ContainerDied","Data":"500aded26b9637724f8922fd51dd0864fabfdac6bd353efbcdfbbe043be389ab"}
Sep 14 02:02:02.142372 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:02.142279    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh" event={"ID":"8d8b6081-a265-4cd4-91ce-0f9686345c82","Type":"ContainerDied","Data":"672bfdc5a9833ae1aba8ffc5f849ac5d34cc574cd2485cab07357bf1f3bdba7f"}
Sep 14 02:02:02.142372 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:02.142298    2581 scope.go:117] "RemoveContainer" containerID="500aded26b9637724f8922fd51dd0864fabfdac6bd353efbcdfbbe043be389ab"
Sep 14 02:02:02.152953 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:02.152932    2581 scope.go:117] "RemoveContainer" containerID="500aded26b9637724f8922fd51dd0864fabfdac6bd353efbcdfbbe043be389ab"
Sep 14 02:02:02.153300 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:02:02.153280    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"500aded26b9637724f8922fd51dd0864fabfdac6bd353efbcdfbbe043be389ab\": container with ID starting with 500aded26b9637724f8922fd51dd0864fabfdac6bd353efbcdfbbe043be389ab not found: ID does not exist" containerID="500aded26b9637724f8922fd51dd0864fabfdac6bd353efbcdfbbe043be389ab"
Sep 14 02:02:02.153351 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:02.153311    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"500aded26b9637724f8922fd51dd0864fabfdac6bd353efbcdfbbe043be389ab"} err="failed to get container status \"500aded26b9637724f8922fd51dd0864fabfdac6bd353efbcdfbbe043be389ab\": rpc error: code = NotFound desc = could not find container \"500aded26b9637724f8922fd51dd0864fabfdac6bd353efbcdfbbe043be389ab\": container with ID starting with 500aded26b9637724f8922fd51dd0864fabfdac6bd353efbcdfbbe043be389ab not found: ID does not exist"
Sep 14 02:02:02.154596 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:02.154577    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:02:02.163501 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:02.163474    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"]
Sep 14 02:02:02.167494 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:02.167466    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-86995c/sub-test-model-86995c-kserve-66b7f95bc9-nzhwh"]
Sep 14 02:02:03.054267 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:03.054230    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d8b6081-a265-4cd4-91ce-0f9686345c82" path="/var/lib/kubelet/pods/8d8b6081-a265-4cd4-91ce-0f9686345c82/volumes"
Sep 14 02:02:21.059283 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:21.059241    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:02:44.059305 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:44.059263    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:02:53.262067 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:02:53.262027    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:03:00.124855 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:00.124819    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"]
Sep 14 02:03:03.556046 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:03.556009    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:03:15.147455 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.144298    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc" podUID="7f242c87-cfd7-4f36-a43c-17d1137186fd" containerName="main" containerID="cri-o://9958bcf54e514001b1ee9ac6d3b6b394a25cadfbf01c8d0b9a812615d9228f22" gracePeriod=45
Sep 14 02:03:15.396135 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.396108    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:03:15.500422 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.500381    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-model-cache\") pod \"7f242c87-cfd7-4f36-a43c-17d1137186fd\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") "
Sep 14 02:03:15.500612 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.500513    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-dshm\") pod \"7f242c87-cfd7-4f36-a43c-17d1137186fd\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") "
Sep 14 02:03:15.500612 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.500543    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg8qj\" (UniqueName: \"kubernetes.io/projected/7f242c87-cfd7-4f36-a43c-17d1137186fd-kube-api-access-mg8qj\") pod \"7f242c87-cfd7-4f36-a43c-17d1137186fd\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") "
Sep 14 02:03:15.500612 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.500586    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f242c87-cfd7-4f36-a43c-17d1137186fd-tls-certs\") pod \"7f242c87-cfd7-4f36-a43c-17d1137186fd\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") "
Sep 14 02:03:15.500780 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.500704    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-home\") pod \"7f242c87-cfd7-4f36-a43c-17d1137186fd\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") "
Sep 14 02:03:15.500780 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.500754    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-tmp-dir\") pod \"7f242c87-cfd7-4f36-a43c-17d1137186fd\" (UID: \"7f242c87-cfd7-4f36-a43c-17d1137186fd\") "
Sep 14 02:03:15.500893 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.500705    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-model-cache" (OuterVolumeSpecName: "model-cache") pod "7f242c87-cfd7-4f36-a43c-17d1137186fd" (UID: "7f242c87-cfd7-4f36-a43c-17d1137186fd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:03:15.500990 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.500898    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-home" (OuterVolumeSpecName: "home") pod "7f242c87-cfd7-4f36-a43c-17d1137186fd" (UID: "7f242c87-cfd7-4f36-a43c-17d1137186fd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:03:15.501095 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.501071    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7f242c87-cfd7-4f36-a43c-17d1137186fd" (UID: "7f242c87-cfd7-4f36-a43c-17d1137186fd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:03:15.501152 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.501092    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:03:15.501152 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.501111    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:03:15.502799 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.502770    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f242c87-cfd7-4f36-a43c-17d1137186fd-kube-api-access-mg8qj" (OuterVolumeSpecName: "kube-api-access-mg8qj") pod "7f242c87-cfd7-4f36-a43c-17d1137186fd" (UID: "7f242c87-cfd7-4f36-a43c-17d1137186fd"). InnerVolumeSpecName "kube-api-access-mg8qj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:03:15.503178 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.503154    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-dshm" (OuterVolumeSpecName: "dshm") pod "7f242c87-cfd7-4f36-a43c-17d1137186fd" (UID: "7f242c87-cfd7-4f36-a43c-17d1137186fd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:03:15.503178 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.503164    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f242c87-cfd7-4f36-a43c-17d1137186fd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7f242c87-cfd7-4f36-a43c-17d1137186fd" (UID: "7f242c87-cfd7-4f36-a43c-17d1137186fd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:03:15.571685 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.571651    2581 generic.go:356] "Generic (PLEG): container finished" podID="7f242c87-cfd7-4f36-a43c-17d1137186fd" containerID="9958bcf54e514001b1ee9ac6d3b6b394a25cadfbf01c8d0b9a812615d9228f22" exitCode=0
Sep 14 02:03:15.571896 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.571697    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"
Sep 14 02:03:15.571896 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.571741    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc" event={"ID":"7f242c87-cfd7-4f36-a43c-17d1137186fd","Type":"ContainerDied","Data":"9958bcf54e514001b1ee9ac6d3b6b394a25cadfbf01c8d0b9a812615d9228f22"}
Sep 14 02:03:15.571896 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.571789    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc" event={"ID":"7f242c87-cfd7-4f36-a43c-17d1137186fd","Type":"ContainerDied","Data":"194e7108e0094533cecf46199c73dfab4d41f621e39331cec60715ec1c1f3aad"}
Sep 14 02:03:15.571896 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.571810    2581 scope.go:117] "RemoveContainer" containerID="9958bcf54e514001b1ee9ac6d3b6b394a25cadfbf01c8d0b9a812615d9228f22"
Sep 14 02:03:15.581442 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.581426    2581 scope.go:117] "RemoveContainer" containerID="9958bcf54e514001b1ee9ac6d3b6b394a25cadfbf01c8d0b9a812615d9228f22"
Sep 14 02:03:15.581733 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:03:15.581713    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9958bcf54e514001b1ee9ac6d3b6b394a25cadfbf01c8d0b9a812615d9228f22\": container with ID starting with 9958bcf54e514001b1ee9ac6d3b6b394a25cadfbf01c8d0b9a812615d9228f22 not found: ID does not exist" containerID="9958bcf54e514001b1ee9ac6d3b6b394a25cadfbf01c8d0b9a812615d9228f22"
Sep 14 02:03:15.581800 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.581741    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9958bcf54e514001b1ee9ac6d3b6b394a25cadfbf01c8d0b9a812615d9228f22"} err="failed to get container status \"9958bcf54e514001b1ee9ac6d3b6b394a25cadfbf01c8d0b9a812615d9228f22\": rpc error: code = NotFound desc = could not find container \"9958bcf54e514001b1ee9ac6d3b6b394a25cadfbf01c8d0b9a812615d9228f22\": container with ID starting with 9958bcf54e514001b1ee9ac6d3b6b394a25cadfbf01c8d0b9a812615d9228f22 not found: ID does not exist"
Sep 14 02:03:15.592355 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.592328    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"]
Sep 14 02:03:15.594635 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.594610    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5d3ac7/rate-test-model-5d3ac7-kserve-7c74bb7fdc-xv8hc"]
Sep 14 02:03:15.602092 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.602075    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:03:15.602092 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.602093    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg8qj\" (UniqueName: \"kubernetes.io/projected/7f242c87-cfd7-4f36-a43c-17d1137186fd-kube-api-access-mg8qj\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:03:15.602221 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.602103    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f242c87-cfd7-4f36-a43c-17d1137186fd-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:03:15.602221 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:15.602113    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f242c87-cfd7-4f36-a43c-17d1137186fd-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:03:16.951413 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:16.951377    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:03:17.055179 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:17.055143    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f242c87-cfd7-4f36-a43c-17d1137186fd" path="/var/lib/kubelet/pods/7f242c87-cfd7-4f36-a43c-17d1137186fd/volumes"
Sep 14 02:03:36.804161 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:36.804076    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"]
Sep 14 02:03:36.804641 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:36.804619    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d8b6081-a265-4cd4-91ce-0f9686345c82" containerName="main"
Sep 14 02:03:36.804641 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:36.804635    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d8b6081-a265-4cd4-91ce-0f9686345c82" containerName="main"
Sep 14 02:03:36.804719 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:36.804644    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f242c87-cfd7-4f36-a43c-17d1137186fd" containerName="main"
Sep 14 02:03:36.804719 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:36.804651    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f242c87-cfd7-4f36-a43c-17d1137186fd" containerName="main"
Sep 14 02:03:36.804792 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:36.804744    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d8b6081-a265-4cd4-91ce-0f9686345c82" containerName="main"
Sep 14 02:03:36.804792 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:36.804755    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f242c87-cfd7-4f36-a43c-17d1137186fd" containerName="main"
Sep 14 02:03:36.814551 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:36.814521    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:03:36.817356 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:36.817330    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-8020f1-serving-cert\""
Sep 14 02:03:36.818027 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:36.817993    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"]
Sep 14 02:03:36.905983 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:36.905939    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96c37483-2098-4d96-885c-c8ca8296b611-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:03:36.906170 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:36.906000    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96c37483-2098-4d96-885c-c8ca8296b611-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:03:36.906170 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:36.906025    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96c37483-2098-4d96-885c-c8ca8296b611-metrics-certs\") pod \"maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:03:36.906170 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:36.906097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4tm9\" (UniqueName: \"kubernetes.io/projected/96c37483-2098-4d96-885c-c8ca8296b611-kube-api-access-c4tm9\") pod \"maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:03:37.006685 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:37.006629    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96c37483-2098-4d96-885c-c8ca8296b611-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:03:37.006685 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:37.006688    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96c37483-2098-4d96-885c-c8ca8296b611-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:03:37.007090 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:37.006727    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96c37483-2098-4d96-885c-c8ca8296b611-metrics-certs\") pod \"maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:03:37.007090 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:37.006809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4tm9\" (UniqueName: \"kubernetes.io/projected/96c37483-2098-4d96-885c-c8ca8296b611-kube-api-access-c4tm9\") pod \"maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:03:37.007540 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:37.007519    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96c37483-2098-4d96-885c-c8ca8296b611-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:03:37.009497 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:37.009466    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96c37483-2098-4d96-885c-c8ca8296b611-metrics-certs\") pod \"maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:03:37.009592 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:37.009523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96c37483-2098-4d96-885c-c8ca8296b611-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:03:37.015681 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:37.015646    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4tm9\" (UniqueName: \"kubernetes.io/projected/96c37483-2098-4d96-885c-c8ca8296b611-kube-api-access-c4tm9\") pod \"maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:03:37.128143 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:37.128050    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:03:37.408587 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:37.408554    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"]
Sep 14 02:03:37.415944 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:37.414600    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 02:03:37.699947 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:37.699816    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh" event={"ID":"96c37483-2098-4d96-885c-c8ca8296b611","Type":"ContainerStarted","Data":"0306cd0e4d6b191ca08e1f8672d6aa2a50427ef17dc4e53d411c948c1837ebb5"}
Sep 14 02:03:38.707757 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:38.707713    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh" event={"ID":"96c37483-2098-4d96-885c-c8ca8296b611","Type":"ContainerStarted","Data":"ea05f8e41fdfdc1584c3e90767b869729f75d1126f1f107018509a13cfaedeee"}
Sep 14 02:03:38.708248 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:38.707883    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:03:38.727049 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:38.726992    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh" podStartSLOduration=2.173614864 podStartE2EDuration="2.726974202s" totalImagesPullingTime="553.359338ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:03:38.724787236 +0000 UTC m=+1978.205238057" watchObservedRunningTime="2026-09-14 02:03:38.726974202 +0000 UTC m=+1978.207425023"
Sep 14 02:03:42.162774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:42.162732    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:03:44.719839 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:03:44.719802    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:04:20.309985 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.309937    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"]
Sep 14 02:04:20.313647 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.313627    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.316544 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.316515    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-39c298\"/\"kube-root-ca.crt\""
Sep 14 02:04:20.316544 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.316535    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-39c298\"/\"ipp-route-39c298-kserve-self-signed-certs\""
Sep 14 02:04:20.316717 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.316585    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-39c298\"/\"default-dockercfg-pkzcb\""
Sep 14 02:04:20.316717 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.316643    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-39c298\"/\"openshift-service-ca.crt\""
Sep 14 02:04:20.320851 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.320819    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"]
Sep 14 02:04:20.415329 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.415292    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-model-cache\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.415329 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.415337    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-home\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.415565 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.415421    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-dshm\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.415565 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.415486    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-tmp-dir\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.415565 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.415535    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e61780-060d-4a0b-aa06-2ccc88cebdce-tls-certs\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.415665 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.415568    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h6tk\" (UniqueName: \"kubernetes.io/projected/92e61780-060d-4a0b-aa06-2ccc88cebdce-kube-api-access-8h6tk\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.516140 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.516102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-dshm\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.516348 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.516163    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-tmp-dir\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.516348 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.516214    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e61780-060d-4a0b-aa06-2ccc88cebdce-tls-certs\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.516348 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.516253    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h6tk\" (UniqueName: \"kubernetes.io/projected/92e61780-060d-4a0b-aa06-2ccc88cebdce-kube-api-access-8h6tk\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.516348 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.516302    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-model-cache\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.516348 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.516330    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-home\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.516635 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.516601    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-tmp-dir\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.516729 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.516708    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-model-cache\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.516779 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.516748    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-home\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.518654 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.518629    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-dshm\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.518872 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.518852    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e61780-060d-4a0b-aa06-2ccc88cebdce-tls-certs\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.523395 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.523370    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h6tk\" (UniqueName: \"kubernetes.io/projected/92e61780-060d-4a0b-aa06-2ccc88cebdce-kube-api-access-8h6tk\") pod \"ipp-route-39c298-kserve-576c744f9f-gg89v\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") " pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.625830 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.625735    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:20.902840 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.902814    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"]
Sep 14 02:04:20.904637 ip-10-0-132-11 kubenswrapper[2581]: W0914 02:04:20.904607    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92e61780_060d_4a0b_aa06_2ccc88cebdce.slice/crio-7086be407b9ebf9d07efe173c38abd2d4e4a07266dbef8f3791fb016a4b8f132 WatchSource:0}: Error finding container 7086be407b9ebf9d07efe173c38abd2d4e4a07266dbef8f3791fb016a4b8f132: Status 404 returned error can't find the container with id 7086be407b9ebf9d07efe173c38abd2d4e4a07266dbef8f3791fb016a4b8f132
Sep 14 02:04:20.950516 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:20.950468    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v" event={"ID":"92e61780-060d-4a0b-aa06-2ccc88cebdce","Type":"ContainerStarted","Data":"7086be407b9ebf9d07efe173c38abd2d4e4a07266dbef8f3791fb016a4b8f132"}
Sep 14 02:04:21.957645 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:21.957605    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v" event={"ID":"92e61780-060d-4a0b-aa06-2ccc88cebdce","Type":"ContainerStarted","Data":"6d9824bd17d0fd549e6bb29220269199763b6c0a41b124fd0df54d38d8df3ca9"}
Sep 14 02:04:21.972502 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:21.972448    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v" podStartSLOduration=1.972431321 podStartE2EDuration="1.972431321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:04:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:04:21.971025004 +0000 UTC m=+2021.451475825" watchObservedRunningTime="2026-09-14 02:04:21.972431321 +0000 UTC m=+2021.452882142"
Sep 14 02:04:24.161687 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:24.161644    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:04:30.626403 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:30.626361    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:30.627064 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:30.626428    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:30.639452 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:30.639421    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:31.017790 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:31.017761    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:04:59.865862 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:04:59.865816    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:05:12.922172 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:12.922083    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"]
Sep 14 02:05:12.926241 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:12.926214    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:12.928081 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:12.928052    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-072cc4-serving-cert\""
Sep 14 02:05:12.933354 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:12.933326    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"]
Sep 14 02:05:13.011807 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:13.011770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:13.012023 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:13.011828    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwppl\" (UniqueName: \"kubernetes.io/projected/52a03445-4c82-45bf-819d-99b380d51e91-kube-api-access-nwppl\") pod \"maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:13.012023 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:13.011907    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:13.012023 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:13.011984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52a03445-4c82-45bf-819d-99b380d51e91-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:13.113478 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:13.113432    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:13.113724 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:13.113507    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52a03445-4c82-45bf-819d-99b380d51e91-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:13.113724 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:13.113587    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:13.113724 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:13.113640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwppl\" (UniqueName: \"kubernetes.io/projected/52a03445-4c82-45bf-819d-99b380d51e91-kube-api-access-nwppl\") pod \"maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:13.113952 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:05:13.113799    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-serving-cert: secret "maas-api-e2e-ipp-cleanup-072cc4-serving-cert" not found
Sep 14 02:05:13.113952 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:05:13.113883    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-maas-api-tls podName:52a03445-4c82-45bf-819d-99b380d51e91 nodeName:}" failed. No retries permitted until 2026-09-14 02:05:13.613860962 +0000 UTC m=+2073.094311763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck" (UID: "52a03445-4c82-45bf-819d-99b380d51e91") : secret "maas-api-e2e-ipp-cleanup-072cc4-serving-cert" not found
Sep 14 02:05:13.114277 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:13.114254    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52a03445-4c82-45bf-819d-99b380d51e91-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:13.116239 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:13.116213    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:13.123664 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:13.123637    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwppl\" (UniqueName: \"kubernetes.io/projected/52a03445-4c82-45bf-819d-99b380d51e91-kube-api-access-nwppl\") pod \"maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:13.616885 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:13.616849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:13.617121 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:05:13.617060    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-serving-cert: secret "maas-api-e2e-ipp-cleanup-072cc4-serving-cert" not found
Sep 14 02:05:13.617192 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:05:13.617155    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-maas-api-tls podName:52a03445-4c82-45bf-819d-99b380d51e91 nodeName:}" failed. No retries permitted until 2026-09-14 02:05:14.617134901 +0000 UTC m=+2074.097585704 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck" (UID: "52a03445-4c82-45bf-819d-99b380d51e91") : secret "maas-api-e2e-ipp-cleanup-072cc4-serving-cert" not found
Sep 14 02:05:14.627164 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:14.627104    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:14.629874 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:14.629845    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:14.743157 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:14.743119    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:15.020363 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:15.020329    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"]
Sep 14 02:05:15.023620 ip-10-0-132-11 kubenswrapper[2581]: W0914 02:05:15.023590    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a03445_4c82_45bf_819d_99b380d51e91.slice/crio-45419b90d132c4049dfb2032a1f185eac7f56b078272f5d26bd19313fc1dd0a9 WatchSource:0}: Error finding container 45419b90d132c4049dfb2032a1f185eac7f56b078272f5d26bd19313fc1dd0a9: Status 404 returned error can't find the container with id 45419b90d132c4049dfb2032a1f185eac7f56b078272f5d26bd19313fc1dd0a9
Sep 14 02:05:15.264670 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:15.264631    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck" event={"ID":"52a03445-4c82-45bf-819d-99b380d51e91","Type":"ContainerStarted","Data":"45419b90d132c4049dfb2032a1f185eac7f56b078272f5d26bd19313fc1dd0a9"}
Sep 14 02:05:16.272029 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:16.271993    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck" event={"ID":"52a03445-4c82-45bf-819d-99b380d51e91","Type":"ContainerStarted","Data":"b5a4ae218fd91b32ee10a17632aa1fec932156646213300c7cd38e4c52d663ed"}
Sep 14 02:05:16.272443 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:16.272060    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:16.287737 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:16.287686    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck" podStartSLOduration=3.705454855 podStartE2EDuration="4.287671257s" totalImagesPullingTime="582.216402ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:05:16.28653731 +0000 UTC m=+2075.766988131" watchObservedRunningTime="2026-09-14 02:05:16.287671257 +0000 UTC m=+2075.768122077"
Sep 14 02:05:19.060063 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:19.060023    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:05:22.284395 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:22.284366    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:41.221377 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:41.221349    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 02:05:41.234512 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:41.234485    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 02:05:51.781975 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:51.781907    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"]
Sep 14 02:05:51.784577 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:51.782181    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck" podUID="52a03445-4c82-45bf-819d-99b380d51e91" containerName="maas-api" containerID="cri-o://b5a4ae218fd91b32ee10a17632aa1fec932156646213300c7cd38e4c52d663ed" gracePeriod=30
Sep 14 02:05:52.039020 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.038949    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:52.174832 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.174801    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-maas-api-tls\") pod \"52a03445-4c82-45bf-819d-99b380d51e91\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") "
Sep 14 02:05:52.175059 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.174851    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-metrics-certs\") pod \"52a03445-4c82-45bf-819d-99b380d51e91\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") "
Sep 14 02:05:52.175059 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.174891    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwppl\" (UniqueName: \"kubernetes.io/projected/52a03445-4c82-45bf-819d-99b380d51e91-kube-api-access-nwppl\") pod \"52a03445-4c82-45bf-819d-99b380d51e91\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") "
Sep 14 02:05:52.175059 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.174983    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52a03445-4c82-45bf-819d-99b380d51e91-openshift-service-ca\") pod \"52a03445-4c82-45bf-819d-99b380d51e91\" (UID: \"52a03445-4c82-45bf-819d-99b380d51e91\") "
Sep 14 02:05:52.175403 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.175371    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52a03445-4c82-45bf-819d-99b380d51e91-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "52a03445-4c82-45bf-819d-99b380d51e91" (UID: "52a03445-4c82-45bf-819d-99b380d51e91"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:05:52.177097 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.177059    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "52a03445-4c82-45bf-819d-99b380d51e91" (UID: "52a03445-4c82-45bf-819d-99b380d51e91"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:05:52.177208 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.177091    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "52a03445-4c82-45bf-819d-99b380d51e91" (UID: "52a03445-4c82-45bf-819d-99b380d51e91"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:05:52.177208 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.177177    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52a03445-4c82-45bf-819d-99b380d51e91-kube-api-access-nwppl" (OuterVolumeSpecName: "kube-api-access-nwppl") pod "52a03445-4c82-45bf-819d-99b380d51e91" (UID: "52a03445-4c82-45bf-819d-99b380d51e91"). InnerVolumeSpecName "kube-api-access-nwppl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:05:52.276715 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.276659    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:05:52.276715 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.276709    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwppl\" (UniqueName: \"kubernetes.io/projected/52a03445-4c82-45bf-819d-99b380d51e91-kube-api-access-nwppl\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:05:52.276715 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.276726    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52a03445-4c82-45bf-819d-99b380d51e91-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:05:52.277034 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.276739    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52a03445-4c82-45bf-819d-99b380d51e91-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:05:52.472347 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.472311    2581 generic.go:356] "Generic (PLEG): container finished" podID="52a03445-4c82-45bf-819d-99b380d51e91" containerID="b5a4ae218fd91b32ee10a17632aa1fec932156646213300c7cd38e4c52d663ed" exitCode=0
Sep 14 02:05:52.472564 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.472362    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"
Sep 14 02:05:52.472564 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.472397    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck" event={"ID":"52a03445-4c82-45bf-819d-99b380d51e91","Type":"ContainerDied","Data":"b5a4ae218fd91b32ee10a17632aa1fec932156646213300c7cd38e4c52d663ed"}
Sep 14 02:05:52.472564 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.472465    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck" event={"ID":"52a03445-4c82-45bf-819d-99b380d51e91","Type":"ContainerDied","Data":"45419b90d132c4049dfb2032a1f185eac7f56b078272f5d26bd19313fc1dd0a9"}
Sep 14 02:05:52.472564 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.472497    2581 scope.go:117] "RemoveContainer" containerID="b5a4ae218fd91b32ee10a17632aa1fec932156646213300c7cd38e4c52d663ed"
Sep 14 02:05:52.483605 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.483585    2581 scope.go:117] "RemoveContainer" containerID="b5a4ae218fd91b32ee10a17632aa1fec932156646213300c7cd38e4c52d663ed"
Sep 14 02:05:52.483967 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:05:52.483944    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5a4ae218fd91b32ee10a17632aa1fec932156646213300c7cd38e4c52d663ed\": container with ID starting with b5a4ae218fd91b32ee10a17632aa1fec932156646213300c7cd38e4c52d663ed not found: ID does not exist" containerID="b5a4ae218fd91b32ee10a17632aa1fec932156646213300c7cd38e4c52d663ed"
Sep 14 02:05:52.484078 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.483981    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5a4ae218fd91b32ee10a17632aa1fec932156646213300c7cd38e4c52d663ed"} err="failed to get container status \"b5a4ae218fd91b32ee10a17632aa1fec932156646213300c7cd38e4c52d663ed\": rpc error: code = NotFound desc = could not find container \"b5a4ae218fd91b32ee10a17632aa1fec932156646213300c7cd38e4c52d663ed\": container with ID starting with b5a4ae218fd91b32ee10a17632aa1fec932156646213300c7cd38e4c52d663ed not found: ID does not exist"
Sep 14 02:05:52.499200 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.499170    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"]
Sep 14 02:05:52.502885 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:52.502855    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-072cc4-79bb8bd8c6-2wcck"]
Sep 14 02:05:53.053869 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:53.053827    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52a03445-4c82-45bf-819d-99b380d51e91" path="/var/lib/kubelet/pods/52a03445-4c82-45bf-819d-99b380d51e91/volumes"
Sep 14 02:05:56.265138 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:05:56.265101    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:06:16.652581 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:16.652547    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:06:24.609283 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:24.609241    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"]
Sep 14 02:06:30.154835 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:30.154797    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:06:39.629132 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.629024    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v" podUID="92e61780-060d-4a0b-aa06-2ccc88cebdce" containerName="main" containerID="cri-o://6d9824bd17d0fd549e6bb29220269199763b6c0a41b124fd0df54d38d8df3ca9" gracePeriod=45
Sep 14 02:06:39.883841 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.883773    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:06:39.934482 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.934444    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-tmp-dir\") pod \"92e61780-060d-4a0b-aa06-2ccc88cebdce\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") "
Sep 14 02:06:39.934689 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.934496    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-model-cache\") pod \"92e61780-060d-4a0b-aa06-2ccc88cebdce\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") "
Sep 14 02:06:39.934689 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.934538    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-dshm\") pod \"92e61780-060d-4a0b-aa06-2ccc88cebdce\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") "
Sep 14 02:06:39.934689 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.934571    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e61780-060d-4a0b-aa06-2ccc88cebdce-tls-certs\") pod \"92e61780-060d-4a0b-aa06-2ccc88cebdce\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") "
Sep 14 02:06:39.934689 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.934632    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-home\") pod \"92e61780-060d-4a0b-aa06-2ccc88cebdce\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") "
Sep 14 02:06:39.934689 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.934650    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8h6tk\" (UniqueName: \"kubernetes.io/projected/92e61780-060d-4a0b-aa06-2ccc88cebdce-kube-api-access-8h6tk\") pod \"92e61780-060d-4a0b-aa06-2ccc88cebdce\" (UID: \"92e61780-060d-4a0b-aa06-2ccc88cebdce\") "
Sep 14 02:06:39.934979 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.934769    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-model-cache" (OuterVolumeSpecName: "model-cache") pod "92e61780-060d-4a0b-aa06-2ccc88cebdce" (UID: "92e61780-060d-4a0b-aa06-2ccc88cebdce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:06:39.934979 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.934776    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "92e61780-060d-4a0b-aa06-2ccc88cebdce" (UID: "92e61780-060d-4a0b-aa06-2ccc88cebdce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:06:39.934979 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.934963    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-home" (OuterVolumeSpecName: "home") pod "92e61780-060d-4a0b-aa06-2ccc88cebdce" (UID: "92e61780-060d-4a0b-aa06-2ccc88cebdce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:06:39.935156 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.934971    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:06:39.935156 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.935012    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:06:39.937036 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.936973    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92e61780-060d-4a0b-aa06-2ccc88cebdce-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "92e61780-060d-4a0b-aa06-2ccc88cebdce" (UID: "92e61780-060d-4a0b-aa06-2ccc88cebdce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:06:39.937165 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.937035    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92e61780-060d-4a0b-aa06-2ccc88cebdce-kube-api-access-8h6tk" (OuterVolumeSpecName: "kube-api-access-8h6tk") pod "92e61780-060d-4a0b-aa06-2ccc88cebdce" (UID: "92e61780-060d-4a0b-aa06-2ccc88cebdce"). InnerVolumeSpecName "kube-api-access-8h6tk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:06:39.937165 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:39.937050    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-dshm" (OuterVolumeSpecName: "dshm") pod "92e61780-060d-4a0b-aa06-2ccc88cebdce" (UID: "92e61780-060d-4a0b-aa06-2ccc88cebdce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:06:40.036242 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:40.036203    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:06:40.036242 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:40.036237    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e61780-060d-4a0b-aa06-2ccc88cebdce-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:06:40.036242 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:40.036247    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e61780-060d-4a0b-aa06-2ccc88cebdce-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:06:40.036519 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:40.036255    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8h6tk\" (UniqueName: \"kubernetes.io/projected/92e61780-060d-4a0b-aa06-2ccc88cebdce-kube-api-access-8h6tk\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:06:40.743363 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:40.743328    2581 generic.go:356] "Generic (PLEG): container finished" podID="92e61780-060d-4a0b-aa06-2ccc88cebdce" containerID="6d9824bd17d0fd549e6bb29220269199763b6c0a41b124fd0df54d38d8df3ca9" exitCode=0
Sep 14 02:06:40.743788 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:40.743372    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"
Sep 14 02:06:40.743788 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:40.743424    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v" event={"ID":"92e61780-060d-4a0b-aa06-2ccc88cebdce","Type":"ContainerDied","Data":"6d9824bd17d0fd549e6bb29220269199763b6c0a41b124fd0df54d38d8df3ca9"}
Sep 14 02:06:40.743788 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:40.743466    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v" event={"ID":"92e61780-060d-4a0b-aa06-2ccc88cebdce","Type":"ContainerDied","Data":"7086be407b9ebf9d07efe173c38abd2d4e4a07266dbef8f3791fb016a4b8f132"}
Sep 14 02:06:40.743788 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:40.743486    2581 scope.go:117] "RemoveContainer" containerID="6d9824bd17d0fd549e6bb29220269199763b6c0a41b124fd0df54d38d8df3ca9"
Sep 14 02:06:40.753671 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:40.753650    2581 scope.go:117] "RemoveContainer" containerID="6d9824bd17d0fd549e6bb29220269199763b6c0a41b124fd0df54d38d8df3ca9"
Sep 14 02:06:40.753991 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:06:40.753972    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d9824bd17d0fd549e6bb29220269199763b6c0a41b124fd0df54d38d8df3ca9\": container with ID starting with 6d9824bd17d0fd549e6bb29220269199763b6c0a41b124fd0df54d38d8df3ca9 not found: ID does not exist" containerID="6d9824bd17d0fd549e6bb29220269199763b6c0a41b124fd0df54d38d8df3ca9"
Sep 14 02:06:40.754053 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:40.754003    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d9824bd17d0fd549e6bb29220269199763b6c0a41b124fd0df54d38d8df3ca9"} err="failed to get container status \"6d9824bd17d0fd549e6bb29220269199763b6c0a41b124fd0df54d38d8df3ca9\": rpc error: code = NotFound desc = could not find container \"6d9824bd17d0fd549e6bb29220269199763b6c0a41b124fd0df54d38d8df3ca9\": container with ID starting with 6d9824bd17d0fd549e6bb29220269199763b6c0a41b124fd0df54d38d8df3ca9 not found: ID does not exist"
Sep 14 02:06:40.764227 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:40.764201    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"]
Sep 14 02:06:40.767313 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:40.767290    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-39c298/ipp-route-39c298-kserve-576c744f9f-gg89v"]
Sep 14 02:06:41.055835 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:06:41.055734    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92e61780-060d-4a0b-aa06-2ccc88cebdce" path="/var/lib/kubelet/pods/92e61780-060d-4a0b-aa06-2ccc88cebdce/volumes"
Sep 14 02:07:18.139455 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.139415    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"]
Sep 14 02:07:18.140012 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.139749    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh" podUID="96c37483-2098-4d96-885c-c8ca8296b611" containerName="maas-api" containerID="cri-o://ea05f8e41fdfdc1584c3e90767b869729f75d1126f1f107018509a13cfaedeee" gracePeriod=30
Sep 14 02:07:18.394186 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.394121    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:07:18.495133 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.495097    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96c37483-2098-4d96-885c-c8ca8296b611-maas-api-tls\") pod \"96c37483-2098-4d96-885c-c8ca8296b611\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") "
Sep 14 02:07:18.495330 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.495146    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96c37483-2098-4d96-885c-c8ca8296b611-metrics-certs\") pod \"96c37483-2098-4d96-885c-c8ca8296b611\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") "
Sep 14 02:07:18.495330 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.495169    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4tm9\" (UniqueName: \"kubernetes.io/projected/96c37483-2098-4d96-885c-c8ca8296b611-kube-api-access-c4tm9\") pod \"96c37483-2098-4d96-885c-c8ca8296b611\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") "
Sep 14 02:07:18.495330 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.495230    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96c37483-2098-4d96-885c-c8ca8296b611-openshift-service-ca\") pod \"96c37483-2098-4d96-885c-c8ca8296b611\" (UID: \"96c37483-2098-4d96-885c-c8ca8296b611\") "
Sep 14 02:07:18.495658 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.495626    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96c37483-2098-4d96-885c-c8ca8296b611-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "96c37483-2098-4d96-885c-c8ca8296b611" (UID: "96c37483-2098-4d96-885c-c8ca8296b611"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:07:18.497380 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.497354    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96c37483-2098-4d96-885c-c8ca8296b611-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "96c37483-2098-4d96-885c-c8ca8296b611" (UID: "96c37483-2098-4d96-885c-c8ca8296b611"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:07:18.497491 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.497390    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96c37483-2098-4d96-885c-c8ca8296b611-kube-api-access-c4tm9" (OuterVolumeSpecName: "kube-api-access-c4tm9") pod "96c37483-2098-4d96-885c-c8ca8296b611" (UID: "96c37483-2098-4d96-885c-c8ca8296b611"). InnerVolumeSpecName "kube-api-access-c4tm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:07:18.497491 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.497447    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96c37483-2098-4d96-885c-c8ca8296b611-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "96c37483-2098-4d96-885c-c8ca8296b611" (UID: "96c37483-2098-4d96-885c-c8ca8296b611"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:07:18.596847 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.596806    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96c37483-2098-4d96-885c-c8ca8296b611-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:07:18.596847 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.596839    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96c37483-2098-4d96-885c-c8ca8296b611-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:07:18.596847 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.596849    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4tm9\" (UniqueName: \"kubernetes.io/projected/96c37483-2098-4d96-885c-c8ca8296b611-kube-api-access-c4tm9\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:07:18.596847 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.596858    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96c37483-2098-4d96-885c-c8ca8296b611-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:07:18.957011 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.956974    2581 generic.go:356] "Generic (PLEG): container finished" podID="96c37483-2098-4d96-885c-c8ca8296b611" containerID="ea05f8e41fdfdc1584c3e90767b869729f75d1126f1f107018509a13cfaedeee" exitCode=0
Sep 14 02:07:18.957197 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.957019    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"
Sep 14 02:07:18.957197 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.957045    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh" event={"ID":"96c37483-2098-4d96-885c-c8ca8296b611","Type":"ContainerDied","Data":"ea05f8e41fdfdc1584c3e90767b869729f75d1126f1f107018509a13cfaedeee"}
Sep 14 02:07:18.957197 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.957086    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh" event={"ID":"96c37483-2098-4d96-885c-c8ca8296b611","Type":"ContainerDied","Data":"0306cd0e4d6b191ca08e1f8672d6aa2a50427ef17dc4e53d411c948c1837ebb5"}
Sep 14 02:07:18.957197 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.957106    2581 scope.go:117] "RemoveContainer" containerID="ea05f8e41fdfdc1584c3e90767b869729f75d1126f1f107018509a13cfaedeee"
Sep 14 02:07:18.967297 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.967279    2581 scope.go:117] "RemoveContainer" containerID="ea05f8e41fdfdc1584c3e90767b869729f75d1126f1f107018509a13cfaedeee"
Sep 14 02:07:18.967614 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:07:18.967588    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea05f8e41fdfdc1584c3e90767b869729f75d1126f1f107018509a13cfaedeee\": container with ID starting with ea05f8e41fdfdc1584c3e90767b869729f75d1126f1f107018509a13cfaedeee not found: ID does not exist" containerID="ea05f8e41fdfdc1584c3e90767b869729f75d1126f1f107018509a13cfaedeee"
Sep 14 02:07:18.967700 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.967620    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea05f8e41fdfdc1584c3e90767b869729f75d1126f1f107018509a13cfaedeee"} err="failed to get container status \"ea05f8e41fdfdc1584c3e90767b869729f75d1126f1f107018509a13cfaedeee\": rpc error: code = NotFound desc = could not find container \"ea05f8e41fdfdc1584c3e90767b869729f75d1126f1f107018509a13cfaedeee\": container with ID starting with ea05f8e41fdfdc1584c3e90767b869729f75d1126f1f107018509a13cfaedeee not found: ID does not exist"
Sep 14 02:07:18.978354 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.978320    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"]
Sep 14 02:07:18.983115 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:18.983085    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8020f1-5bd85cdd56-smkgh"]
Sep 14 02:07:19.054831 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:19.054794    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96c37483-2098-4d96-885c-c8ca8296b611" path="/var/lib/kubelet/pods/96c37483-2098-4d96-885c-c8ca8296b611/volumes"
Sep 14 02:07:21.257798 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:21.257761    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:07:35.284454 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.284407    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"]
Sep 14 02:07:35.285087 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.284714    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4" podUID="b0f292d1-9579-43e8-9c16-752da3c80f30" containerName="maas-api" containerID="cri-o://b10b7be5292a49910c2919937b98c79c747814f26ce5a6c36861c1972ecb4242" gracePeriod=30
Sep 14 02:07:35.311181 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.311148    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d3ac7-29822520-45ltd"]
Sep 14 02:07:35.313252 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.313230    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5d3ac7-29822520-45ltd"]
Sep 14 02:07:35.536345 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.536281    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 02:07:35.650799 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.650764    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68fhd\" (UniqueName: \"kubernetes.io/projected/b0f292d1-9579-43e8-9c16-752da3c80f30-kube-api-access-68fhd\") pod \"b0f292d1-9579-43e8-9c16-752da3c80f30\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") "
Sep 14 02:07:35.650799 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.650804    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0f292d1-9579-43e8-9c16-752da3c80f30-metrics-certs\") pod \"b0f292d1-9579-43e8-9c16-752da3c80f30\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") "
Sep 14 02:07:35.651064 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.650885    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0f292d1-9579-43e8-9c16-752da3c80f30-maas-api-tls\") pod \"b0f292d1-9579-43e8-9c16-752da3c80f30\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") "
Sep 14 02:07:35.651064 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.650933    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0f292d1-9579-43e8-9c16-752da3c80f30-openshift-service-ca\") pod \"b0f292d1-9579-43e8-9c16-752da3c80f30\" (UID: \"b0f292d1-9579-43e8-9c16-752da3c80f30\") "
Sep 14 02:07:35.651395 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.651364    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0f292d1-9579-43e8-9c16-752da3c80f30-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b0f292d1-9579-43e8-9c16-752da3c80f30" (UID: "b0f292d1-9579-43e8-9c16-752da3c80f30"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:07:35.653243 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.653211    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0f292d1-9579-43e8-9c16-752da3c80f30-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b0f292d1-9579-43e8-9c16-752da3c80f30" (UID: "b0f292d1-9579-43e8-9c16-752da3c80f30"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:07:35.653561 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.653536    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0f292d1-9579-43e8-9c16-752da3c80f30-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b0f292d1-9579-43e8-9c16-752da3c80f30" (UID: "b0f292d1-9579-43e8-9c16-752da3c80f30"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:07:35.653561 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.653551    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0f292d1-9579-43e8-9c16-752da3c80f30-kube-api-access-68fhd" (OuterVolumeSpecName: "kube-api-access-68fhd") pod "b0f292d1-9579-43e8-9c16-752da3c80f30" (UID: "b0f292d1-9579-43e8-9c16-752da3c80f30"). InnerVolumeSpecName "kube-api-access-68fhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:07:35.752383 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.752332    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0f292d1-9579-43e8-9c16-752da3c80f30-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:07:35.752383 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.752378    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0f292d1-9579-43e8-9c16-752da3c80f30-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:07:35.752383 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.752400    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68fhd\" (UniqueName: \"kubernetes.io/projected/b0f292d1-9579-43e8-9c16-752da3c80f30-kube-api-access-68fhd\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:07:35.752659 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:35.752414    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0f292d1-9579-43e8-9c16-752da3c80f30-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:07:36.059549 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:36.059512    2581 generic.go:356] "Generic (PLEG): container finished" podID="b0f292d1-9579-43e8-9c16-752da3c80f30" containerID="b10b7be5292a49910c2919937b98c79c747814f26ce5a6c36861c1972ecb4242" exitCode=0
Sep 14 02:07:36.059549 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:36.059550    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"
Sep 14 02:07:36.059774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:36.059599    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4" event={"ID":"b0f292d1-9579-43e8-9c16-752da3c80f30","Type":"ContainerDied","Data":"b10b7be5292a49910c2919937b98c79c747814f26ce5a6c36861c1972ecb4242"}
Sep 14 02:07:36.059774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:36.059656    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4" event={"ID":"b0f292d1-9579-43e8-9c16-752da3c80f30","Type":"ContainerDied","Data":"8dd1ded9d64d64520a085cec029e38045af831601876434b4ccca1864d500b8b"}
Sep 14 02:07:36.059774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:36.059679    2581 scope.go:117] "RemoveContainer" containerID="b10b7be5292a49910c2919937b98c79c747814f26ce5a6c36861c1972ecb4242"
Sep 14 02:07:36.070346 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:36.070328    2581 scope.go:117] "RemoveContainer" containerID="b10b7be5292a49910c2919937b98c79c747814f26ce5a6c36861c1972ecb4242"
Sep 14 02:07:36.070666 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:07:36.070644    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b10b7be5292a49910c2919937b98c79c747814f26ce5a6c36861c1972ecb4242\": container with ID starting with b10b7be5292a49910c2919937b98c79c747814f26ce5a6c36861c1972ecb4242 not found: ID does not exist" containerID="b10b7be5292a49910c2919937b98c79c747814f26ce5a6c36861c1972ecb4242"
Sep 14 02:07:36.070747 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:36.070676    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b10b7be5292a49910c2919937b98c79c747814f26ce5a6c36861c1972ecb4242"} err="failed to get container status \"b10b7be5292a49910c2919937b98c79c747814f26ce5a6c36861c1972ecb4242\": rpc error: code = NotFound desc = could not find container \"b10b7be5292a49910c2919937b98c79c747814f26ce5a6c36861c1972ecb4242\": container with ID starting with b10b7be5292a49910c2919937b98c79c747814f26ce5a6c36861c1972ecb4242 not found: ID does not exist"
Sep 14 02:07:36.081153 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:36.081124    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"]
Sep 14 02:07:36.086181 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:36.086157    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5d3ac7-5dbc44d95b-2dfm4"]
Sep 14 02:07:37.053361 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:37.053327    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b1ba428-224b-4137-9e5d-12d96139be4e" path="/var/lib/kubelet/pods/6b1ba428-224b-4137-9e5d-12d96139be4e/volumes"
Sep 14 02:07:37.053761 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:37.053701    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0f292d1-9579-43e8-9c16-752da3c80f30" path="/var/lib/kubelet/pods/b0f292d1-9579-43e8-9c16-752da3c80f30/volumes"
Sep 14 02:07:38.263611 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:38.263575    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:07:41.431996 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:41.431967    2581 scope.go:117] "RemoveContainer" containerID="2cef8dc740c8322f521b05e8580e6d10dd9d24f558274372c6c586275f4e3ea0"
Sep 14 02:07:55.392017 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:07:55.391975    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:08:03.961801 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:08:03.956090    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:09:26.955628 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:09:26.955585    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:09:51.058167 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:09:51.058089    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:09:58.657766 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:09:58.657725    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:10:06.958372 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:10:06.958339    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:10:25.153269 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:10:25.153235    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:10:41.258279 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:10:41.258253    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 02:10:41.274810 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:10:41.274786    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 02:11:54.061858 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:11:54.061827    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:12:00.474544 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:00.474507    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:12:07.889399 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:07.889360    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:12:25.872540 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:25.872497    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:12:32.809193 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.809161    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"]
Sep 14 02:12:32.809593 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.809554    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92e61780-060d-4a0b-aa06-2ccc88cebdce" containerName="main"
Sep 14 02:12:32.809593 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.809566    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="92e61780-060d-4a0b-aa06-2ccc88cebdce" containerName="main"
Sep 14 02:12:32.809593 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.809576    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c37483-2098-4d96-885c-c8ca8296b611" containerName="maas-api"
Sep 14 02:12:32.809593 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.809583    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c37483-2098-4d96-885c-c8ca8296b611" containerName="maas-api"
Sep 14 02:12:32.809593 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.809591    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f292d1-9579-43e8-9c16-752da3c80f30" containerName="maas-api"
Sep 14 02:12:32.809593 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.809596    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f292d1-9579-43e8-9c16-752da3c80f30" containerName="maas-api"
Sep 14 02:12:32.809790 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.809604    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52a03445-4c82-45bf-819d-99b380d51e91" containerName="maas-api"
Sep 14 02:12:32.809790 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.809609    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="52a03445-4c82-45bf-819d-99b380d51e91" containerName="maas-api"
Sep 14 02:12:32.809790 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.809665    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0f292d1-9579-43e8-9c16-752da3c80f30" containerName="maas-api"
Sep 14 02:12:32.809790 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.809674    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="92e61780-060d-4a0b-aa06-2ccc88cebdce" containerName="main"
Sep 14 02:12:32.809790 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.809681    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="52a03445-4c82-45bf-819d-99b380d51e91" containerName="maas-api"
Sep 14 02:12:32.809790 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.809687    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="96c37483-2098-4d96-885c-c8ca8296b611" containerName="maas-api"
Sep 14 02:12:32.820768 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.820731    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"]
Sep 14 02:12:32.820941 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.820828    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.823526 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.823505    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-106ff464-kserve-self-signed-certs\""
Sep 14 02:12:32.868220 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.868182    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98dfk\" (UniqueName: \"kubernetes.io/projected/fc038725-ca12-4d92-9e9b-5e011790709c-kube-api-access-98dfk\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.868220 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.868228    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc038725-ca12-4d92-9e9b-5e011790709c-tls-certs\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.868443 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.868271    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-model-cache\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.868443 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.868326    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-home\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.868443 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.868346    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-tmp-dir\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.868443 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.868419    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-dshm\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.969812 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.969768    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-dshm\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.970011 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.969828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98dfk\" (UniqueName: \"kubernetes.io/projected/fc038725-ca12-4d92-9e9b-5e011790709c-kube-api-access-98dfk\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.970011 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.969868    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc038725-ca12-4d92-9e9b-5e011790709c-tls-certs\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.970011 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.969902    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-model-cache\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.970011 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.969985    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-home\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.970011 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.970008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-tmp-dir\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.970416 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.970384    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-model-cache\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.970558 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.970450    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-home\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.970558 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.970466    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-tmp-dir\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.972289 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.972267    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-dshm\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.972532 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.972514    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc038725-ca12-4d92-9e9b-5e011790709c-tls-certs\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:32.976810 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:32.976792    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98dfk\" (UniqueName: \"kubernetes.io/projected/fc038725-ca12-4d92-9e9b-5e011790709c-kube-api-access-98dfk\") pod \"e2e-same-id-b-106ff464-kserve-745779df67-5rpvn\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") " pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:33.130820 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:33.130718    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:33.406704 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:33.406674    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"]
Sep 14 02:12:33.408299 ip-10-0-132-11 kubenswrapper[2581]: W0914 02:12:33.408271    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc038725_ca12_4d92_9e9b_5e011790709c.slice/crio-5e00a9ce5cecf4e5cc90a658618cda9057306e1ebd7011e80c5ba51ae39af3ea WatchSource:0}: Error finding container 5e00a9ce5cecf4e5cc90a658618cda9057306e1ebd7011e80c5ba51ae39af3ea: Status 404 returned error can't find the container with id 5e00a9ce5cecf4e5cc90a658618cda9057306e1ebd7011e80c5ba51ae39af3ea
Sep 14 02:12:33.719724 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:33.719690    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn" event={"ID":"fc038725-ca12-4d92-9e9b-5e011790709c","Type":"ContainerStarted","Data":"95b5833ab4344695a9910fc37d7bc7f732f25dbadf1795be26294f6bb4e1741b"}
Sep 14 02:12:33.719724 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:33.719731    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn" event={"ID":"fc038725-ca12-4d92-9e9b-5e011790709c","Type":"ContainerStarted","Data":"5e00a9ce5cecf4e5cc90a658618cda9057306e1ebd7011e80c5ba51ae39af3ea"}
Sep 14 02:12:33.736870 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:33.736810    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn" podStartSLOduration=1.736789367 podStartE2EDuration="1.736789367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:12:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:12:33.736496243 +0000 UTC m=+2513.216947064" watchObservedRunningTime="2026-09-14 02:12:33.736789367 +0000 UTC m=+2513.217240191"
Sep 14 02:12:35.561703 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:35.561665    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:12:43.131341 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:43.131302    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:43.131341 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:43.131351    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:43.144107 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:43.144081    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:43.786979 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:43.786944    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:12:51.115360 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:51.115319    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"]
Sep 14 02:12:53.562799 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:12:53.562762    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:13:01.366196 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:01.366158    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:13:06.142013 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.141958    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn" podUID="fc038725-ca12-4d92-9e9b-5e011790709c" containerName="main" containerID="cri-o://95b5833ab4344695a9910fc37d7bc7f732f25dbadf1795be26294f6bb4e1741b" gracePeriod=45
Sep 14 02:13:06.398114 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.398045    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:13:06.584133 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.584100    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98dfk\" (UniqueName: \"kubernetes.io/projected/fc038725-ca12-4d92-9e9b-5e011790709c-kube-api-access-98dfk\") pod \"fc038725-ca12-4d92-9e9b-5e011790709c\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") "
Sep 14 02:13:06.584346 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.584171    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-home\") pod \"fc038725-ca12-4d92-9e9b-5e011790709c\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") "
Sep 14 02:13:06.584346 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.584235    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-model-cache\") pod \"fc038725-ca12-4d92-9e9b-5e011790709c\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") "
Sep 14 02:13:06.584346 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.584262    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-tmp-dir\") pod \"fc038725-ca12-4d92-9e9b-5e011790709c\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") "
Sep 14 02:13:06.584346 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.584307    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-dshm\") pod \"fc038725-ca12-4d92-9e9b-5e011790709c\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") "
Sep 14 02:13:06.584346 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.584331    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc038725-ca12-4d92-9e9b-5e011790709c-tls-certs\") pod \"fc038725-ca12-4d92-9e9b-5e011790709c\" (UID: \"fc038725-ca12-4d92-9e9b-5e011790709c\") "
Sep 14 02:13:06.584611 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.584513    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-model-cache" (OuterVolumeSpecName: "model-cache") pod "fc038725-ca12-4d92-9e9b-5e011790709c" (UID: "fc038725-ca12-4d92-9e9b-5e011790709c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:13:06.584611 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.584526    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-home" (OuterVolumeSpecName: "home") pod "fc038725-ca12-4d92-9e9b-5e011790709c" (UID: "fc038725-ca12-4d92-9e9b-5e011790709c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:13:06.584611 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.584535    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fc038725-ca12-4d92-9e9b-5e011790709c" (UID: "fc038725-ca12-4d92-9e9b-5e011790709c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:13:06.584762 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.584676    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-model-cache\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:13:06.584762 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.584695    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-tmp-dir\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:13:06.584762 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.584707    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-home\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:13:06.586576 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.586549    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc038725-ca12-4d92-9e9b-5e011790709c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fc038725-ca12-4d92-9e9b-5e011790709c" (UID: "fc038725-ca12-4d92-9e9b-5e011790709c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:13:06.586702 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.586613    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc038725-ca12-4d92-9e9b-5e011790709c-kube-api-access-98dfk" (OuterVolumeSpecName: "kube-api-access-98dfk") pod "fc038725-ca12-4d92-9e9b-5e011790709c" (UID: "fc038725-ca12-4d92-9e9b-5e011790709c"). InnerVolumeSpecName "kube-api-access-98dfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:13:06.586702 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.586613    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-dshm" (OuterVolumeSpecName: "dshm") pod "fc038725-ca12-4d92-9e9b-5e011790709c" (UID: "fc038725-ca12-4d92-9e9b-5e011790709c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:13:06.685614 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.685503    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc038725-ca12-4d92-9e9b-5e011790709c-dshm\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:13:06.685614 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.685552    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc038725-ca12-4d92-9e9b-5e011790709c-tls-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:13:06.685614 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.685563    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98dfk\" (UniqueName: \"kubernetes.io/projected/fc038725-ca12-4d92-9e9b-5e011790709c-kube-api-access-98dfk\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:13:06.899295 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.899253    2581 generic.go:356] "Generic (PLEG): container finished" podID="fc038725-ca12-4d92-9e9b-5e011790709c" containerID="95b5833ab4344695a9910fc37d7bc7f732f25dbadf1795be26294f6bb4e1741b" exitCode=0
Sep 14 02:13:06.899295 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.899300    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"
Sep 14 02:13:06.899506 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.899344    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn" event={"ID":"fc038725-ca12-4d92-9e9b-5e011790709c","Type":"ContainerDied","Data":"95b5833ab4344695a9910fc37d7bc7f732f25dbadf1795be26294f6bb4e1741b"}
Sep 14 02:13:06.899506 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.899397    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn" event={"ID":"fc038725-ca12-4d92-9e9b-5e011790709c","Type":"ContainerDied","Data":"5e00a9ce5cecf4e5cc90a658618cda9057306e1ebd7011e80c5ba51ae39af3ea"}
Sep 14 02:13:06.899506 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.899417    2581 scope.go:117] "RemoveContainer" containerID="95b5833ab4344695a9910fc37d7bc7f732f25dbadf1795be26294f6bb4e1741b"
Sep 14 02:13:06.909684 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.909664    2581 scope.go:117] "RemoveContainer" containerID="95b5833ab4344695a9910fc37d7bc7f732f25dbadf1795be26294f6bb4e1741b"
Sep 14 02:13:06.910019 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:13:06.910000    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95b5833ab4344695a9910fc37d7bc7f732f25dbadf1795be26294f6bb4e1741b\": container with ID starting with 95b5833ab4344695a9910fc37d7bc7f732f25dbadf1795be26294f6bb4e1741b not found: ID does not exist" containerID="95b5833ab4344695a9910fc37d7bc7f732f25dbadf1795be26294f6bb4e1741b"
Sep 14 02:13:06.910079 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.910029    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95b5833ab4344695a9910fc37d7bc7f732f25dbadf1795be26294f6bb4e1741b"} err="failed to get container status \"95b5833ab4344695a9910fc37d7bc7f732f25dbadf1795be26294f6bb4e1741b\": rpc error: code = NotFound desc = could not find container \"95b5833ab4344695a9910fc37d7bc7f732f25dbadf1795be26294f6bb4e1741b\": container with ID starting with 95b5833ab4344695a9910fc37d7bc7f732f25dbadf1795be26294f6bb4e1741b not found: ID does not exist"
Sep 14 02:13:06.920678 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.920650    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"]
Sep 14 02:13:06.925963 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:06.925936    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-106ff464-kserve-745779df67-5rpvn"]
Sep 14 02:13:07.055091 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:07.055028    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc038725-ca12-4d92-9e9b-5e011790709c" path="/var/lib/kubelet/pods/fc038725-ca12-4d92-9e9b-5e011790709c/volumes"
Sep 14 02:13:09.130160 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.127439    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"]
Sep 14 02:13:09.130160 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.128273    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc038725-ca12-4d92-9e9b-5e011790709c" containerName="main"
Sep 14 02:13:09.130160 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.128295    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc038725-ca12-4d92-9e9b-5e011790709c" containerName="main"
Sep 14 02:13:09.130160 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.128463    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc038725-ca12-4d92-9e9b-5e011790709c" containerName="main"
Sep 14 02:13:09.134169 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.134137    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:09.137036 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.137002    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 02:13:09.137177 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.137039    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 02:13:09.137177 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.137078    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 14 02:13:09.137320 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.137247    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 02:13:09.137320 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.137265    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"]
Sep 14 02:13:09.137320 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.137313    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8hsv9\""
Sep 14 02:13:09.308087 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.308045    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-maas-api-tls\") pod \"maas-api-7c69887854-kzn4v\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") " pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:09.308286 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.308159    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-metrics-certs\") pod \"maas-api-7c69887854-kzn4v\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") " pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:09.308286 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.308209    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrg7s\" (UniqueName: \"kubernetes.io/projected/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-kube-api-access-zrg7s\") pod \"maas-api-7c69887854-kzn4v\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") " pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:09.308286 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.308230    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-openshift-service-ca\") pod \"maas-api-7c69887854-kzn4v\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") " pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:09.408878 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.408777    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-metrics-certs\") pod \"maas-api-7c69887854-kzn4v\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") " pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:09.408878 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.408837    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrg7s\" (UniqueName: \"kubernetes.io/projected/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-kube-api-access-zrg7s\") pod \"maas-api-7c69887854-kzn4v\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") " pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:09.408878 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.408858    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-openshift-service-ca\") pod \"maas-api-7c69887854-kzn4v\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") " pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:09.409220 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.408899    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-maas-api-tls\") pod \"maas-api-7c69887854-kzn4v\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") " pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:09.409655 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.409633    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-openshift-service-ca\") pod \"maas-api-7c69887854-kzn4v\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") " pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:09.411442 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.411421    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-metrics-certs\") pod \"maas-api-7c69887854-kzn4v\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") " pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:09.411545 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.411494    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-maas-api-tls\") pod \"maas-api-7c69887854-kzn4v\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") " pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:09.418744 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.418721    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrg7s\" (UniqueName: \"kubernetes.io/projected/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-kube-api-access-zrg7s\") pod \"maas-api-7c69887854-kzn4v\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") " pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:09.446100 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.446057    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:09.453084 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.453046    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:13:09.720834 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.720809    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"]
Sep 14 02:13:09.722530 ip-10-0-132-11 kubenswrapper[2581]: W0914 02:13:09.722504    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefdfbf01_42c9_4a12_a8ac_ecd594c73a3d.slice/crio-f81973fbbaa9679e6d9740ff75668623b7fb2463299e3f3a072be97f1637ed4b WatchSource:0}: Error finding container f81973fbbaa9679e6d9740ff75668623b7fb2463299e3f3a072be97f1637ed4b: Status 404 returned error can't find the container with id f81973fbbaa9679e6d9740ff75668623b7fb2463299e3f3a072be97f1637ed4b
Sep 14 02:13:09.724028 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.724007    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 02:13:09.918653 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:09.918611    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v" event={"ID":"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d","Type":"ContainerStarted","Data":"f81973fbbaa9679e6d9740ff75668623b7fb2463299e3f3a072be97f1637ed4b"}
Sep 14 02:13:10.925884 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:10.925841    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v" event={"ID":"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d","Type":"ContainerStarted","Data":"3bcd89174c2443fd198d9fa8d7364dea0bcaa86cd543ed883ebd07bb0e970f03"}
Sep 14 02:13:10.926323 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:10.926042    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:10.942220 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:10.942163    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v" podStartSLOduration=1.553557565 podStartE2EDuration="1.942076706s" totalImagesPullingTime="388.519141ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:13:10.941498116 +0000 UTC m=+2550.421948939" watchObservedRunningTime="2026-09-14 02:13:10.942076706 +0000 UTC m=+2550.422527527"
Sep 14 02:13:16.297149 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.297114    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"]
Sep 14 02:13:16.297555 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.297297    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v" podUID="efdfbf01-42c9-4a12-a8ac-ecd594c73a3d" containerName="maas-api" containerID="cri-o://3bcd89174c2443fd198d9fa8d7364dea0bcaa86cd543ed883ebd07bb0e970f03" gracePeriod=30
Sep 14 02:13:16.302808 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.302783    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:16.327171 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.327142    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"]
Sep 14 02:13:16.330846 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.330826    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:16.345511 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.345477    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"]
Sep 14 02:13:16.373728 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.373695    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/701d5156-787c-4d1a-b491-280a18c3fcd1-metrics-certs\") pod \"maas-api-794f95dd98-mnmcx\" (UID: \"701d5156-787c-4d1a-b491-280a18c3fcd1\") " pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:16.373880 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.373752    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvmzc\" (UniqueName: \"kubernetes.io/projected/701d5156-787c-4d1a-b491-280a18c3fcd1-kube-api-access-vvmzc\") pod \"maas-api-794f95dd98-mnmcx\" (UID: \"701d5156-787c-4d1a-b491-280a18c3fcd1\") " pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:16.373939 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.373873    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/701d5156-787c-4d1a-b491-280a18c3fcd1-openshift-service-ca\") pod \"maas-api-794f95dd98-mnmcx\" (UID: \"701d5156-787c-4d1a-b491-280a18c3fcd1\") " pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:16.373939 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.373903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/701d5156-787c-4d1a-b491-280a18c3fcd1-maas-api-tls\") pod \"maas-api-794f95dd98-mnmcx\" (UID: \"701d5156-787c-4d1a-b491-280a18c3fcd1\") " pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:16.474727 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.474688    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/701d5156-787c-4d1a-b491-280a18c3fcd1-openshift-service-ca\") pod \"maas-api-794f95dd98-mnmcx\" (UID: \"701d5156-787c-4d1a-b491-280a18c3fcd1\") " pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:16.474901 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.474742    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/701d5156-787c-4d1a-b491-280a18c3fcd1-maas-api-tls\") pod \"maas-api-794f95dd98-mnmcx\" (UID: \"701d5156-787c-4d1a-b491-280a18c3fcd1\") " pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:16.474901 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.474789    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/701d5156-787c-4d1a-b491-280a18c3fcd1-metrics-certs\") pod \"maas-api-794f95dd98-mnmcx\" (UID: \"701d5156-787c-4d1a-b491-280a18c3fcd1\") " pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:16.474901 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.474848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvmzc\" (UniqueName: \"kubernetes.io/projected/701d5156-787c-4d1a-b491-280a18c3fcd1-kube-api-access-vvmzc\") pod \"maas-api-794f95dd98-mnmcx\" (UID: \"701d5156-787c-4d1a-b491-280a18c3fcd1\") " pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:16.475614 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.475583    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/701d5156-787c-4d1a-b491-280a18c3fcd1-openshift-service-ca\") pod \"maas-api-794f95dd98-mnmcx\" (UID: \"701d5156-787c-4d1a-b491-280a18c3fcd1\") " pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:16.478421 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.478359    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/701d5156-787c-4d1a-b491-280a18c3fcd1-metrics-certs\") pod \"maas-api-794f95dd98-mnmcx\" (UID: \"701d5156-787c-4d1a-b491-280a18c3fcd1\") " pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:16.478548 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.478489    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/701d5156-787c-4d1a-b491-280a18c3fcd1-maas-api-tls\") pod \"maas-api-794f95dd98-mnmcx\" (UID: \"701d5156-787c-4d1a-b491-280a18c3fcd1\") " pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:16.484251 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.484227    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvmzc\" (UniqueName: \"kubernetes.io/projected/701d5156-787c-4d1a-b491-280a18c3fcd1-kube-api-access-vvmzc\") pod \"maas-api-794f95dd98-mnmcx\" (UID: \"701d5156-787c-4d1a-b491-280a18c3fcd1\") " pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:16.549094 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.549031    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:16.564492 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.564456    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:13:16.575323 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.575295    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-maas-api-tls\") pod \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") "
Sep 14 02:13:16.575476 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.575343    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrg7s\" (UniqueName: \"kubernetes.io/projected/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-kube-api-access-zrg7s\") pod \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") "
Sep 14 02:13:16.575476 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.575368    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-metrics-certs\") pod \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") "
Sep 14 02:13:16.575476 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.575432    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-openshift-service-ca\") pod \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\" (UID: \"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d\") "
Sep 14 02:13:16.575887 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.575856    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "efdfbf01-42c9-4a12-a8ac-ecd594c73a3d" (UID: "efdfbf01-42c9-4a12-a8ac-ecd594c73a3d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:13:16.578110 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.578088    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "efdfbf01-42c9-4a12-a8ac-ecd594c73a3d" (UID: "efdfbf01-42c9-4a12-a8ac-ecd594c73a3d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:13:16.578180 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.578102    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-kube-api-access-zrg7s" (OuterVolumeSpecName: "kube-api-access-zrg7s") pod "efdfbf01-42c9-4a12-a8ac-ecd594c73a3d" (UID: "efdfbf01-42c9-4a12-a8ac-ecd594c73a3d"). InnerVolumeSpecName "kube-api-access-zrg7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:13:16.578230 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.578200    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "efdfbf01-42c9-4a12-a8ac-ecd594c73a3d" (UID: "efdfbf01-42c9-4a12-a8ac-ecd594c73a3d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:13:16.676836 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.676777    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-openshift-service-ca\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:13:16.676836 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.676828    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-maas-api-tls\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:13:16.676836 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.676839    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrg7s\" (UniqueName: \"kubernetes.io/projected/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-kube-api-access-zrg7s\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:13:16.676836 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.676847    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d-metrics-certs\") on node \"ip-10-0-132-11.ec2.internal\" DevicePath \"\""
Sep 14 02:13:16.684677 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.684654    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:16.958649 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.958619    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"]
Sep 14 02:13:16.960340 ip-10-0-132-11 kubenswrapper[2581]: W0914 02:13:16.960310    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod701d5156_787c_4d1a_b491_280a18c3fcd1.slice/crio-360745c04a97ac814b3820c9ad1352dfc8f22f425e8bb5eb5548b064faa3c668 WatchSource:0}: Error finding container 360745c04a97ac814b3820c9ad1352dfc8f22f425e8bb5eb5548b064faa3c668: Status 404 returned error can't find the container with id 360745c04a97ac814b3820c9ad1352dfc8f22f425e8bb5eb5548b064faa3c668
Sep 14 02:13:16.961494 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.961465    2581 generic.go:356] "Generic (PLEG): container finished" podID="efdfbf01-42c9-4a12-a8ac-ecd594c73a3d" containerID="3bcd89174c2443fd198d9fa8d7364dea0bcaa86cd543ed883ebd07bb0e970f03" exitCode=0
Sep 14 02:13:16.961601 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.961518    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"
Sep 14 02:13:16.961601 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.961547    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v" event={"ID":"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d","Type":"ContainerDied","Data":"3bcd89174c2443fd198d9fa8d7364dea0bcaa86cd543ed883ebd07bb0e970f03"}
Sep 14 02:13:16.961738 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.961607    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7c69887854-kzn4v" event={"ID":"efdfbf01-42c9-4a12-a8ac-ecd594c73a3d","Type":"ContainerDied","Data":"f81973fbbaa9679e6d9740ff75668623b7fb2463299e3f3a072be97f1637ed4b"}
Sep 14 02:13:16.961738 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.961642    2581 scope.go:117] "RemoveContainer" containerID="3bcd89174c2443fd198d9fa8d7364dea0bcaa86cd543ed883ebd07bb0e970f03"
Sep 14 02:13:16.970822 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.970802    2581 scope.go:117] "RemoveContainer" containerID="3bcd89174c2443fd198d9fa8d7364dea0bcaa86cd543ed883ebd07bb0e970f03"
Sep 14 02:13:16.971183 ip-10-0-132-11 kubenswrapper[2581]: E0914 02:13:16.971163    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bcd89174c2443fd198d9fa8d7364dea0bcaa86cd543ed883ebd07bb0e970f03\": container with ID starting with 3bcd89174c2443fd198d9fa8d7364dea0bcaa86cd543ed883ebd07bb0e970f03 not found: ID does not exist" containerID="3bcd89174c2443fd198d9fa8d7364dea0bcaa86cd543ed883ebd07bb0e970f03"
Sep 14 02:13:16.971266 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.971191    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bcd89174c2443fd198d9fa8d7364dea0bcaa86cd543ed883ebd07bb0e970f03"} err="failed to get container status \"3bcd89174c2443fd198d9fa8d7364dea0bcaa86cd543ed883ebd07bb0e970f03\": rpc error: code = NotFound desc = could not find container \"3bcd89174c2443fd198d9fa8d7364dea0bcaa86cd543ed883ebd07bb0e970f03\": container with ID starting with 3bcd89174c2443fd198d9fa8d7364dea0bcaa86cd543ed883ebd07bb0e970f03 not found: ID does not exist"
Sep 14 02:13:16.982429 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.982404    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"]
Sep 14 02:13:16.985560 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:16.985536    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7c69887854-kzn4v"]
Sep 14 02:13:17.053955 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:17.053898    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efdfbf01-42c9-4a12-a8ac-ecd594c73a3d" path="/var/lib/kubelet/pods/efdfbf01-42c9-4a12-a8ac-ecd594c73a3d/volumes"
Sep 14 02:13:17.969490 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:17.969450    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx" event={"ID":"701d5156-787c-4d1a-b491-280a18c3fcd1","Type":"ContainerStarted","Data":"18d849c2cadd528d43279ada8e336dca9bd823c1ac093119f3063978fa052020"}
Sep 14 02:13:17.969968 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:17.969517    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx" event={"ID":"701d5156-787c-4d1a-b491-280a18c3fcd1","Type":"ContainerStarted","Data":"360745c04a97ac814b3820c9ad1352dfc8f22f425e8bb5eb5548b064faa3c668"}
Sep 14 02:13:17.969968 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:17.969652    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:17.986036 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:17.985986    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx" podStartSLOduration=1.665936251 podStartE2EDuration="1.985967959s" totalImagesPullingTime="320.031708ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:13:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:13:17.98458183 +0000 UTC m=+2557.465032652" watchObservedRunningTime="2026-09-14 02:13:17.985967959 +0000 UTC m=+2557.466418793"
Sep 14 02:13:23.257489 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:23.257448    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:13:23.982387 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:23.982358    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-794f95dd98-mnmcx"
Sep 14 02:13:28.960762 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:28.960721    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:13:36.455602 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:36.455565    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-tm7z5"]
Sep 14 02:13:38.895445 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:38.895403    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5c44878777-wbhzm_71c7e401-c498-4d13-b69b-6a4d718ee4a6/authorino/0.log"
Sep 14 02:13:44.648078 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:44.648043    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c5d46b944-lbcsc_2674dd1c-d5b3-427c-9386-bc375c7268e7/copy-manifests/0.log"
Sep 14 02:13:44.653333 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:44.653310    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c5d46b944-lbcsc_2674dd1c-d5b3-427c-9386-bc375c7268e7/manager/0.log"
Sep 14 02:13:45.097264 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:45.097232    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-794f95dd98-mnmcx_701d5156-787c-4d1a-b491-280a18c3fcd1/maas-api/0.log"
Sep 14 02:13:46.243036 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:46.243001    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5c44878777-wbhzm_71c7e401-c498-4d13-b69b-6a4d718ee4a6/authorino/0.log"
Sep 14 02:13:46.553106 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:46.553022    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-q2lpf_4c2de50f-539a-434c-acd0-a7ed170f55e9/kuadrant-console-plugin/0.log"
Sep 14 02:13:46.797824 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:46.797792    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-tm7z5_ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58/limitador/0.log"
Sep 14 02:13:47.559805 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:47.559778    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cb7b7df59-zgxzz_fc761bf5-2d66-475f-8c59-3b81cb4d8fb2/kube-auth-proxy/0.log"
Sep 14 02:13:48.008904 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:48.008874    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-84988c94c6-fp8j5_ec2956f5-bbef-4b16-bdb6-6f70058685b2/router/0.log"
Sep 14 02:13:48.661772 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:48.661742    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-k2x95_6f4e7b9a-52f2-414b-a479-7ac5e92977dd/main/0.log"
Sep 14 02:13:48.869735 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:48.869704    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-clwbc_7e6b3d50-75f6-4ddf-a8c8-c4ad3a1769fd/main/0.log"
Sep 14 02:13:48.976007 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:48.975979    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-gxxp2_c72667e5-4930-432d-b8fc-7e5ae7202b0c/main/0.log"
Sep 14 02:13:57.593686 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:57.593655    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dw759_7cc6e131-076a-4f42-bdc9-874e49b5b938/global-pull-secret-syncer/0.log"
Sep 14 02:13:57.743306 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:57.743277    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wb7hr_86a1cb60-daa6-4531-b896-ea13966bd97c/konnectivity-agent/0.log"
Sep 14 02:13:57.784283 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:13:57.784248    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-11.ec2.internal_9fd5c0f6fc359fb985fe321e9822e84e/haproxy/0.log"
Sep 14 02:14:02.566211 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:02.566118    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5c44878777-wbhzm_71c7e401-c498-4d13-b69b-6a4d718ee4a6/authorino/0.log"
Sep 14 02:14:02.682239 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:02.682212    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-q2lpf_4c2de50f-539a-434c-acd0-a7ed170f55e9/kuadrant-console-plugin/0.log"
Sep 14 02:14:02.745217 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:02.745186    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-tm7z5_ac8c2ae0-32cb-4a1c-b516-728b5fcc0d58/limitador/0.log"
Sep 14 02:14:04.544550 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:04.544522    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-r5l49_63e51821-9401-42bc-8ecc-db50bae06d09/cluster-monitoring-operator/0.log"
Sep 14 02:14:04.697000 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:04.696965    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-gdtkp_3e8b1fdd-1050-4b1e-8de5-11b9c4fd68a1/monitoring-plugin/0.log"
Sep 14 02:14:04.744695 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:04.744661    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lbzvd_de2f4016-4c8f-40c7-ab01-f65c559befde/node-exporter/0.log"
Sep 14 02:14:04.765311 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:04.765235    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lbzvd_de2f4016-4c8f-40c7-ab01-f65c559befde/kube-rbac-proxy/0.log"
Sep 14 02:14:04.785756 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:04.785733    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lbzvd_de2f4016-4c8f-40c7-ab01-f65c559befde/init-textfile/0.log"
Sep 14 02:14:04.965239 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:04.965211    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-cpq8r_b2ff7849-fec9-4c2f-aef9-74830987667a/kube-rbac-proxy-main/0.log"
Sep 14 02:14:04.986211 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:04.986182    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-cpq8r_b2ff7849-fec9-4c2f-aef9-74830987667a/kube-rbac-proxy-self/0.log"
Sep 14 02:14:05.009608 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:05.009568    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-cpq8r_b2ff7849-fec9-4c2f-aef9-74830987667a/openshift-state-metrics/0.log"
Sep 14 02:14:05.230956 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:05.230929    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-zb45z_8e613f4c-b402-4660-b0b9-6aab3a126440/prometheus-operator/0.log"
Sep 14 02:14:05.250269 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:05.250243    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-zb45z_8e613f4c-b402-4660-b0b9-6aab3a126440/kube-rbac-proxy/0.log"
Sep 14 02:14:05.381700 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:05.381672    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cb589b864-gk2vw_9ee86a74-4f23-4850-94cf-0af95a3a5361/thanos-query/0.log"
Sep 14 02:14:05.404002 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:05.403969    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cb589b864-gk2vw_9ee86a74-4f23-4850-94cf-0af95a3a5361/kube-rbac-proxy-web/0.log"
Sep 14 02:14:05.426746 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:05.426717    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cb589b864-gk2vw_9ee86a74-4f23-4850-94cf-0af95a3a5361/kube-rbac-proxy/0.log"
Sep 14 02:14:05.451251 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:05.451218    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cb589b864-gk2vw_9ee86a74-4f23-4850-94cf-0af95a3a5361/prom-label-proxy/0.log"
Sep 14 02:14:05.471425 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:05.471402    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cb589b864-gk2vw_9ee86a74-4f23-4850-94cf-0af95a3a5361/kube-rbac-proxy-rules/0.log"
Sep 14 02:14:05.492688 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:05.492608    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-cb589b864-gk2vw_9ee86a74-4f23-4850-94cf-0af95a3a5361/kube-rbac-proxy-metrics/0.log"
Sep 14 02:14:06.877759 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:06.877726    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-xhr5r_9935e3f9-36eb-4f10-9022-9a2699cf0866/networking-console-plugin/0.log"
Sep 14 02:14:07.888008 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:07.887971    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"]
Sep 14 02:14:07.888571 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:07.888428    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efdfbf01-42c9-4a12-a8ac-ecd594c73a3d" containerName="maas-api"
Sep 14 02:14:07.888571 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:07.888448    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="efdfbf01-42c9-4a12-a8ac-ecd594c73a3d" containerName="maas-api"
Sep 14 02:14:07.888661 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:07.888605    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="efdfbf01-42c9-4a12-a8ac-ecd594c73a3d" containerName="maas-api"
Sep 14 02:14:07.891716 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:07.891697    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:07.893869 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:07.893846    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7nzg8\"/\"openshift-service-ca.crt\""
Sep 14 02:14:07.893869 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:07.893864    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7nzg8\"/\"kube-root-ca.crt\""
Sep 14 02:14:07.894072 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:07.893892    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7nzg8\"/\"default-dockercfg-nvqmh\""
Sep 14 02:14:07.900206 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:07.900182    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"]
Sep 14 02:14:07.950814 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:07.950761    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8d742947-8943-496b-a60b-acefd59bded3-proc\") pod \"perf-node-gather-daemonset-h2zmr\" (UID: \"8d742947-8943-496b-a60b-acefd59bded3\") " pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:07.951031 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:07.950874    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d742947-8943-496b-a60b-acefd59bded3-sys\") pod \"perf-node-gather-daemonset-h2zmr\" (UID: \"8d742947-8943-496b-a60b-acefd59bded3\") " pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:07.951031 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:07.950933    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88xbv\" (UniqueName: \"kubernetes.io/projected/8d742947-8943-496b-a60b-acefd59bded3-kube-api-access-88xbv\") pod \"perf-node-gather-daemonset-h2zmr\" (UID: \"8d742947-8943-496b-a60b-acefd59bded3\") " pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:07.951031 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:07.950997    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d742947-8943-496b-a60b-acefd59bded3-lib-modules\") pod \"perf-node-gather-daemonset-h2zmr\" (UID: \"8d742947-8943-496b-a60b-acefd59bded3\") " pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:07.951150 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:07.951038    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8d742947-8943-496b-a60b-acefd59bded3-podres\") pod \"perf-node-gather-daemonset-h2zmr\" (UID: \"8d742947-8943-496b-a60b-acefd59bded3\") " pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:08.052271 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:08.052233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88xbv\" (UniqueName: \"kubernetes.io/projected/8d742947-8943-496b-a60b-acefd59bded3-kube-api-access-88xbv\") pod \"perf-node-gather-daemonset-h2zmr\" (UID: \"8d742947-8943-496b-a60b-acefd59bded3\") " pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:08.052460 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:08.052288    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d742947-8943-496b-a60b-acefd59bded3-lib-modules\") pod \"perf-node-gather-daemonset-h2zmr\" (UID: \"8d742947-8943-496b-a60b-acefd59bded3\") " pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:08.052460 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:08.052325    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8d742947-8943-496b-a60b-acefd59bded3-podres\") pod \"perf-node-gather-daemonset-h2zmr\" (UID: \"8d742947-8943-496b-a60b-acefd59bded3\") " pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:08.052460 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:08.052381    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8d742947-8943-496b-a60b-acefd59bded3-proc\") pod \"perf-node-gather-daemonset-h2zmr\" (UID: \"8d742947-8943-496b-a60b-acefd59bded3\") " pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:08.052460 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:08.052456    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d742947-8943-496b-a60b-acefd59bded3-sys\") pod \"perf-node-gather-daemonset-h2zmr\" (UID: \"8d742947-8943-496b-a60b-acefd59bded3\") " pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:08.052663 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:08.052459    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d742947-8943-496b-a60b-acefd59bded3-lib-modules\") pod \"perf-node-gather-daemonset-h2zmr\" (UID: \"8d742947-8943-496b-a60b-acefd59bded3\") " pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:08.052663 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:08.052531    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8d742947-8943-496b-a60b-acefd59bded3-podres\") pod \"perf-node-gather-daemonset-h2zmr\" (UID: \"8d742947-8943-496b-a60b-acefd59bded3\") " pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:08.052663 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:08.052546    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8d742947-8943-496b-a60b-acefd59bded3-proc\") pod \"perf-node-gather-daemonset-h2zmr\" (UID: \"8d742947-8943-496b-a60b-acefd59bded3\") " pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:08.052663 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:08.052552    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d742947-8943-496b-a60b-acefd59bded3-sys\") pod \"perf-node-gather-daemonset-h2zmr\" (UID: \"8d742947-8943-496b-a60b-acefd59bded3\") " pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:08.059103 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:08.059079    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88xbv\" (UniqueName: \"kubernetes.io/projected/8d742947-8943-496b-a60b-acefd59bded3-kube-api-access-88xbv\") pod \"perf-node-gather-daemonset-h2zmr\" (UID: \"8d742947-8943-496b-a60b-acefd59bded3\") " pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:08.071254 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:08.071236    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-79894c44d8-lck7n_1a154ab1-6eb3-4772-810c-8beb7b4ee52c/console/0.log"
Sep 14 02:14:08.202532 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:08.202432    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:08.473452 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:08.473392    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"]
Sep 14 02:14:08.475872 ip-10-0-132-11 kubenswrapper[2581]: W0914 02:14:08.475842    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8d742947_8943_496b_a60b_acefd59bded3.slice/crio-8b69baa59913d24ba911293b794c66830afbb71076a0a1691655be3a176f87e3 WatchSource:0}: Error finding container 8b69baa59913d24ba911293b794c66830afbb71076a0a1691655be3a176f87e3: Status 404 returned error can't find the container with id 8b69baa59913d24ba911293b794c66830afbb71076a0a1691655be3a176f87e3
Sep 14 02:14:09.255467 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:09.255434    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr" event={"ID":"8d742947-8943-496b-a60b-acefd59bded3","Type":"ContainerStarted","Data":"f4e2177bc24d25dc0d99b706be1cbb6142e0fb48542a6da4aaf55608d5bc38e3"}
Sep 14 02:14:09.255892 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:09.255484    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr" event={"ID":"8d742947-8943-496b-a60b-acefd59bded3","Type":"ContainerStarted","Data":"8b69baa59913d24ba911293b794c66830afbb71076a0a1691655be3a176f87e3"}
Sep 14 02:14:09.255892 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:09.255511    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:09.269749 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:09.269697    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr" podStartSLOduration=2.269681741 podStartE2EDuration="2.269681741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:14:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:14:09.269017288 +0000 UTC m=+2608.749468109" watchObservedRunningTime="2026-09-14 02:14:09.269681741 +0000 UTC m=+2608.750132562"
Sep 14 02:14:09.836205 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:09.836179    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jxz4h_248acebf-b6bf-407e-ad54-0976487a1b2c/dns/0.log"
Sep 14 02:14:09.857655 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:09.857617    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jxz4h_248acebf-b6bf-407e-ad54-0976487a1b2c/kube-rbac-proxy/0.log"
Sep 14 02:14:09.944641 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:09.944614    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bzb4b_4ab20462-366d-4e07-a302-25be0aef9dd9/dns-node-resolver/0.log"
Sep 14 02:14:10.563437 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:10.563402    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rlnvn_f00644fe-4d6e-49e3-a485-eb4029ad78ad/node-ca/0.log"
Sep 14 02:14:12.104267 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:12.104240    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cb7b7df59-zgxzz_fc761bf5-2d66-475f-8c59-3b81cb4d8fb2/kube-auth-proxy/0.log"
Sep 14 02:14:12.216167 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:12.216136    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-84988c94c6-fp8j5_ec2956f5-bbef-4b16-bdb6-6f70058685b2/router/0.log"
Sep 14 02:14:13.060289 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:13.060257    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5fs9z_74829c0c-ae15-4b20-b653-1e23c44c899b/serve-healthcheck-canary/0.log"
Sep 14 02:14:13.554767 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:13.554727    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-crc4t_624bd393-bc43-4a14-a0de-cf8ba209b0ed/insights-operator/0.log"
Sep 14 02:14:13.555238 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:13.555113    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-crc4t_624bd393-bc43-4a14-a0de-cf8ba209b0ed/insights-operator/1.log"
Sep 14 02:14:13.574154 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:13.574127    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2cl5j_d062c861-cb50-4bdd-bac9-7eba1d0a74bd/kube-rbac-proxy/0.log"
Sep 14 02:14:13.594806 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:13.594779    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2cl5j_d062c861-cb50-4bdd-bac9-7eba1d0a74bd/exporter/0.log"
Sep 14 02:14:13.614863 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:13.614836    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2cl5j_d062c861-cb50-4bdd-bac9-7eba1d0a74bd/extractor/0.log"
Sep 14 02:14:15.270392 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:15.270364    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7nzg8/perf-node-gather-daemonset-h2zmr"
Sep 14 02:14:16.312157 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:16.312127    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c5d46b944-lbcsc_2674dd1c-d5b3-427c-9386-bc375c7268e7/manager/0.log"
Sep 14 02:14:16.332245 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:16.332219    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c5d46b944-lbcsc_2674dd1c-d5b3-427c-9386-bc375c7268e7/copy-manifests/0.log"
Sep 14 02:14:17.577078 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:17.577047    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5f97d9f44d-4sj9c_513bbf17-afe2-495b-bffa-fc9cb2e23918/manager/0.log"
Sep 14 02:14:18.452171 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:18.452086    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cb7b7df59-zgxzz_fc761bf5-2d66-475f-8c59-3b81cb4d8fb2/kube-auth-proxy/0.log"
Sep 14 02:14:18.541720 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:18.541687    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-84988c94c6-fp8j5_ec2956f5-bbef-4b16-bdb6-6f70058685b2/router/0.log"
Sep 14 02:14:20.868938 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:20.868886    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cb7b7df59-zgxzz_fc761bf5-2d66-475f-8c59-3b81cb4d8fb2/kube-auth-proxy/0.log"
Sep 14 02:14:20.958417 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:20.958374    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-84988c94c6-fp8j5_ec2956f5-bbef-4b16-bdb6-6f70058685b2/router/0.log"
Sep 14 02:14:24.708774 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:24.708744    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ktprp_4e0a5004-422d-425e-954f-1d2bcba81a37/kube-multus-additional-cni-plugins/0.log"
Sep 14 02:14:24.728257 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:24.728233    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ktprp_4e0a5004-422d-425e-954f-1d2bcba81a37/egress-router-binary-copy/0.log"
Sep 14 02:14:24.747521 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:24.747493    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ktprp_4e0a5004-422d-425e-954f-1d2bcba81a37/cni-plugins/0.log"
Sep 14 02:14:24.767151 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:24.767125    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ktprp_4e0a5004-422d-425e-954f-1d2bcba81a37/bond-cni-plugin/0.log"
Sep 14 02:14:24.786422 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:24.786399    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ktprp_4e0a5004-422d-425e-954f-1d2bcba81a37/routeoverride-cni/0.log"
Sep 14 02:14:24.805995 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:24.805968    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ktprp_4e0a5004-422d-425e-954f-1d2bcba81a37/whereabouts-cni-bincopy/0.log"
Sep 14 02:14:24.826049 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:24.826016    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ktprp_4e0a5004-422d-425e-954f-1d2bcba81a37/whereabouts-cni/0.log"
Sep 14 02:14:24.879335 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:24.879303    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fcn4g_c10b8a1c-148a-4fdc-86f2-42c2aa6b298b/kube-multus/0.log"
Sep 14 02:14:25.001724 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:25.001694    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fv6nk_ff2b27ad-c198-489c-9bb8-befeea447a64/network-metrics-daemon/0.log"
Sep 14 02:14:25.019162 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:25.019134    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fv6nk_ff2b27ad-c198-489c-9bb8-befeea447a64/kube-rbac-proxy/0.log"
Sep 14 02:14:25.879096 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:25.879066    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-controller/0.log"
Sep 14 02:14:25.893861 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:25.893835    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/1.log"
Sep 14 02:14:25.893861 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:25.893860    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovn-acl-logging/0.log"
Sep 14 02:14:25.914825 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:25.914794    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/kube-rbac-proxy-node/0.log"
Sep 14 02:14:25.934652 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:25.934621    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/kube-rbac-proxy-ovn-metrics/0.log"
Sep 14 02:14:25.950039 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:25.950017    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/northd/0.log"
Sep 14 02:14:25.969235 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:25.969212    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/nbdb/0.log"
Sep 14 02:14:25.988692 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:25.988663    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/sbdb/0.log"
Sep 14 02:14:26.043860 ip-10-0-132-11 kubenswrapper[2581]: I0914 02:14:26.043827    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gf7k5_dcc0a7d3-0591-4558-899a-0950f6920814/ovnkube-controller/0.log"