Sep 10 17:39:09.384999 ip-10-0-139-245 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 17:39:09.385009 ip-10-0-139-245 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 17:39:09.385016 ip-10-0-139-245 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 17:39:09.385274 ip-10-0-139-245 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 17:39:19.540638 ip-10-0-139-245 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 17:39:19.540654 ip-10-0-139-245 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 595bfab9731643b7a08459388a34a0e7 --
Sep 10 17:41:15.397017 ip-10-0-139-245 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 17:41:15.804545 ip-10-0-139-245 kubenswrapper[2549]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 17:41:15.804545 ip-10-0-139-245 kubenswrapper[2549]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 17:41:15.804545 ip-10-0-139-245 kubenswrapper[2549]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 17:41:15.804545 ip-10-0-139-245 kubenswrapper[2549]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 17:41:15.804545 ip-10-0-139-245 kubenswrapper[2549]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 17:41:15.806501 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.806410    2549 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 17:41:15.812349 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812323    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:41:15.812349 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812344    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:41:15.812349 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812350    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:41:15.812349 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812354    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:41:15.812349 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812357    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812360    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812363    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812367    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812371    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812375    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812379    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812382    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812385    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812388    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812391    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812394    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812397    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812400    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812403    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812405    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812408    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812411    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812414    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:41:15.812554 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812416    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812419    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812421    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812424    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812427    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812430    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812433    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812438    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812441    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812444    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812447    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812451    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812455    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812457    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812461    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812464    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812466    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812469    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812471    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:41:15.813044 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812474    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812477    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812480    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812482    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812485    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812488    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812491    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812493    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812496    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812499    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812501    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812504    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812506    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812509    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812511    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812514    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812516    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812519    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812521    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812524    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:41:15.813518 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812527    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812530    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812532    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812535    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812538    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812541    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812544    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812547    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812551    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812553    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812556    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812559    2549 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812562    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812565    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812567    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812570    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812573    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812575    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812578    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812580    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:41:15.814031 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812583    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812586    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812589    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812592    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812594    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812597    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812599    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812602    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812604    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812607    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812609    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812612    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812615    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812617    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812620    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812622    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.812625    2549 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813374    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813382    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813386    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:41:15.814557 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813389    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813392    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813395    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813398    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813401    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813404    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813407    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813409    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813412    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813415    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813418    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813420    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813423    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813425    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813428    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813431    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813434    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813436    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813439    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813442    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:41:15.815101 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813445    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813447    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813450    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813453    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813455    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813458    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813460    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813463    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813466    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813468    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813471    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813474    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813477    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813480    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813483    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813485    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813488    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813490    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813495    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:41:15.815622 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813499    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813502    2549 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813505    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813508    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813511    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813514    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813516    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813519    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813522    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813525    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813528    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813531    2549 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813533    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813536    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813539    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813541    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813544    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813546    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813549    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813551    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:41:15.816134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813554    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813559    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813562    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813566    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813570    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813573    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813575    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813578    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813581    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813584    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813587    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813590    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813593    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813597    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813600    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813603    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813607    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813610    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813612    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:41:15.816636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813615    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813620    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813624    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813627    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813630    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813632    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813635    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813638    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813641    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813643    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813646    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813649    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813652    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813654    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813657    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813659    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813665    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.813667    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814084    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814090    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:41:15.817134 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814093    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814096    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814098    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814101    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814104    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814106    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814109    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814111    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814114    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814117    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814120    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814122    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814124    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814127    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814130    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814133    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814136    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814139    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814141    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814144    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:41:15.817650 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814146    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814149    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814151    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814154    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814156    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814159    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814163    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814166    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814171    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814174    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814176    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814179    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814181    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814184    2549 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814188    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814192    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814196    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814199    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814201    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:41:15.818260 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814203    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814206    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814209    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814211    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814214    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814217    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814220    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814222    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814225    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814228    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814230    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814233    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814236    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814238    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814241    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814243    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814246    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814248    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814251    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814254    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814256    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:41:15.818759 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814261    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814263    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814266    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814269    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814271    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814274    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814276    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814279    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814281    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814284    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814286    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814289    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814292    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814294    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814298    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814302    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814304    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814307    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814310    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:41:15.819328 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814313    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814318    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814320    2549 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814323    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814325    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814328    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814331    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814333    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814336    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814338    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814341    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814343    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814346    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814350    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814353    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814356    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814358    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.814361    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814436    2549 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814481    2549 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814490    2549 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 17:41:15.819808 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814494    2549 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814499    2549 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814502    2549 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814507    2549 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814512    2549 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814515    2549 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814518    2549 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814522    2549 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814525    2549 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814528    2549 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814531    2549 flags.go:64] FLAG: --cgroup-root=""
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814534    2549 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814537    2549 flags.go:64] FLAG: --client-ca-file=""
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814541    2549 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814544    2549 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814548    2549 flags.go:64] FLAG: --cluster-domain=""
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814551    2549 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814555    2549 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814558    2549 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814562    2549 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814565    2549 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814568    2549 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814572    2549 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 17:41:15.820365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814575    2549 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814579    2549 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814583    2549 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814585    2549 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814589    2549 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814592    2549 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814596    2549 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814599    2549 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814602    2549 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814605    2549 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814608    2549 flags.go:64] FLAG: --enable-server="true"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814611    2549 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814615    2549 flags.go:64] FLAG: --event-burst="100"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814618    2549 flags.go:64] FLAG: --event-qps="50"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814621    2549 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814624    2549 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814627    2549 flags.go:64] FLAG: --eviction-hard=""
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814631    2549 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814634    2549 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814637    2549 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814640    2549 flags.go:64] FLAG: --eviction-soft=""
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814643    2549 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814647    2549 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814650    2549 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814653    2549 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 17:41:15.820949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814656    2549 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814659    2549 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814662    2549 flags.go:64] FLAG: --feature-gates=""
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814666    2549 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814669    2549 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814672    2549 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814675    2549 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814678    2549 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814683    2549 flags.go:64] FLAG: --help="false"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814686    2549 flags.go:64] FLAG: --hostname-override="ip-10-0-139-245.ec2.internal"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814704    2549 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814708    2549 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814711    2549 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814714    2549 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814718    2549 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814721    2549 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814724    2549 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814727    2549 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814730    2549 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814733    2549 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814736    2549 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814739    2549 flags.go:64] FLAG: --kube-reserved=""
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814742    2549 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814745    2549 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 17:41:15.821598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814748    2549 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814751    2549 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814754    2549 flags.go:64] FLAG: --lock-file=""
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814756    2549 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814759    2549 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814762    2549 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814768    2549 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814772    2549 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814775    2549 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814778    2549 flags.go:64] FLAG: --logging-format="text"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814781    2549 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814784    2549 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814787    2549 flags.go:64] FLAG: --manifest-url=""
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814790    2549 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814794    2549 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814797    2549 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814803    2549 flags.go:64] FLAG: --max-pods="110"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814806    2549 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814809    2549 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814812    2549 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814814    2549 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814818    2549 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814821    2549 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814824    2549 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814832    2549 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 17:41:15.822226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814835    2549 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814838    2549 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814841    2549 flags.go:64] FLAG: --pod-cidr=""
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814844    2549 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814850    2549 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814853    2549 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814856    2549 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814859    2549 flags.go:64] FLAG: --port="10250"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814862    2549 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814865    2549 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f68fbdad1cfb018d"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814868    2549 flags.go:64] FLAG: --qos-reserved=""
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814871    2549 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814874    2549 flags.go:64] FLAG: --register-node="true"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814877    2549 flags.go:64] FLAG: --register-with-taints=""
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814881    2549 flags.go:64] FLAG: --registry-burst="10"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814884    2549 flags.go:64] FLAG: --registry-qps="5"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814888    2549 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814891    2549 flags.go:64] FLAG: --reserved-memory=""
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814894    2549 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814898    2549 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814901    2549 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814904    2549 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814906    2549 flags.go:64] FLAG: --runonce="false"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814909    2549 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814915    2549 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 17:41:15.822884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814918    2549 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814921    2549 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814924    2549 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814927    2549 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814930    2549 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814934    2549 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814937    2549 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814939    2549 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814942    2549 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814945    2549 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814948    2549 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814951    2549 flags.go:64] FLAG: --system-cgroups=""
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814954    2549 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814960    2549 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814963    2549 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814966    2549 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814970    2549 flags.go:64] FLAG: --tls-min-version=""
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814973    2549 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814975    2549 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814978    2549 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814981    2549 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814986    2549 flags.go:64] FLAG: --v="2"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.814990    2549 flags.go:64] FLAG: --version="false"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.815001    2549 flags.go:64] FLAG: --vmodule=""
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.815005    2549 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 17:41:15.823494 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.815008    2549 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815128    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815131    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815135    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815138    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815141    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815145    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815148    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815151    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815154    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815157    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815160    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815163    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815166    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815169    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815172    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815175    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815178    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815180    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815183    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:41:15.824163 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815187    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815191    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815194    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815197    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815199    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815201    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815204    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815207    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815211    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815214    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815216    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815220    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815222    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815225    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815227    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815230    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815232    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815235    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815238    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815241    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:41:15.824659 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815243    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815246    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815248    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815251    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815254    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815256    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815259    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815262    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815264    2549 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815267    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815269    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815272    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815275    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815277    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815280    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815282    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815285    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815289    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815293    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:41:15.825194 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815296    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815300    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815303    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815305    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815308    2549 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815312    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815315    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815318    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815321    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815323    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815326    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815330    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815333    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815335    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815338    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815342    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815345    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815348    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815351    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815354    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:41:15.825673 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815357    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815360    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815363    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815366    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815369    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815372    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815375    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815378    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815380    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815382    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815385    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815387    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815390    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815394    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815396    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815399    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815401    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815404    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815407    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815410    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:41:15.826200 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.815412    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:41:15.826801 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.816053    2549 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 17:41:15.826801 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.823524    2549 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 17:41:15.826801 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.823542    2549 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 17:41:15.826801 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.823561    2549 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 17:41:15.826801 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.823567    2549 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 17:41:15.826801 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823627    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:41:15.826801 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823632    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:41:15.826801 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823636    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:41:15.826801 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823639    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:41:15.826801 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823642    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:41:15.826801 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823645    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:41:15.826801 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823648    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:41:15.826801 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823651    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:41:15.826801 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823655    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:41:15.826801 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823657    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823660    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823662    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823665    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823668    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823671    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823673    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823676    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823680    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823684    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823687    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823705    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823709    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823713    2549 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823716    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823719    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823722    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823725    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823728    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823730    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:41:15.827229 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823735    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823738    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823741    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823743    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823746    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823749    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823751    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823754    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823757    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823759    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823762    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823764    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823767    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823770    2549 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823772    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823775    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823779    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823781    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823784    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823787    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:41:15.827734 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823789    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823792    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823794    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823797    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823799    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823802    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823804    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823807    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823810    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823812    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823814    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823817    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823821    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823824    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823827    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823829    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823832    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823834    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823837    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823839    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:41:15.828225 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823843    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823846    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823848    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823851    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823854    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823856    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823858    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823861    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823864    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823866    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823870    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823874    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823877    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823879    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823882    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823886    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823889    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823892    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823895    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:41:15.828727 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823898    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:41:15.829196 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823901    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:41:15.829196 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823903    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:41:15.829196 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823906    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:41:15.829196 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823909    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:41:15.829196 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823912    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:41:15.829196 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823915    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:41:15.829196 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823919    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:41:15.829196 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823922    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:41:15.829196 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823924    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:41:15.829196 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.823927    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:41:15.829196 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.823933    2549 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 17:41:15.829196 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824057    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:41:15.829196 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824063    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:41:15.829196 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824066    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824069    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824072    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824075    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824077    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824080    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824083    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824085    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824089    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824092    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824095    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824099    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824103    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824105    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824108    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824111    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824114    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824117    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824120    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:41:15.829562 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824122    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824125    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824127    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824130    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824133    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824136    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824138    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824141    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824144    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824147    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824149    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824152    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824154    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824157    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824161    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824164    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824168    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824171    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824174    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:41:15.830050 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824177    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824179    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824182    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824185    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824188    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824190    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824193    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824195    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824198    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824200    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824203    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824205    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824207    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824210    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824213    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824215    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824218    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824221    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824223    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824226    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:41:15.830546 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824228    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824232    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824235    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824237    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824239    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824242    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824245    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824247    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824250    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824252    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824255    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824257    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824260    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824262    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824264    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824267    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824270    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824272    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824275    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824277    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:41:15.831140 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824280    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824282    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824285    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824287    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824290    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824292    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824295    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824297    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824300    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824302    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824305    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824308    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824311    2549 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824313    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824316    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824319    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824323    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824326    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:41:15.831636 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:15.824328    2549 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:41:15.832092 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.824333    2549 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 17:41:15.832092 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.824461    2549 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 17:41:15.832092 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.826686    2549 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 17:41:15.832092 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.827601    2549 server.go:1019] "Starting client certificate rotation"
Sep 10 17:41:15.832092 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.827708    2549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 17:41:15.832092 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.828367    2549 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 17:41:15.832092 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.830255    2549 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 17:41:15.833062 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.833045    2549 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 17:41:15.843740 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.843716    2549 log.go:25] "Validated CRI v1 runtime API"
Sep 10 17:41:15.846798 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.846782    2549 log.go:25] "Validated CRI v1 image API"
Sep 10 17:41:15.847630 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.847614    2549 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 17:41:15.850318 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.850296    2549 fs.go:135] Filesystem UUIDs: map[24e24dc0-2371-49fc-9233-0a5ad92dfcbb:/dev/nvme0n1p3 402b772e-fa43-4aeb-844b-f58bd7ba0509:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 10 17:41:15.850390 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.850316    2549 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 17:41:15.856702 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.856566    2549 manager.go:216] Machine: {Timestamp:2026-09-10 17:41:15.85527277 +0000 UTC m=+0.346324569 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096309 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2eff3af9dc49d63ca8480bf8ba18fa SystemUUID:ec2eff3a-f9dc-49d6-3ca8-480bf8ba18fa BootID:595bfab9-7316-43b7-a084-59388a34a0e7 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:28:08:ef:1d:37 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:28:08:ef:1d:37 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:e3:69:ec:41:db Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 17:41:15.856702 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.856688    2549 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 17:41:15.856826 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.856799    2549 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 17:41:15.858869 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.858851    2549 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 17:41:15.860135 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.860110    2549 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 17:41:15.860288 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.860141    2549 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-245.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 17:41:15.861417 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.861406    2549 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 17:41:15.861452 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.861419    2549 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 17:41:15.861452 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.861428    2549 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 17:41:15.861986 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.861975    2549 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 17:41:15.863289 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.863279    2549 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 17:41:15.867029 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.867016    2549 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 17:41:15.867160 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.867150    2549 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 17:41:15.867237 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.867181    2549 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 17:41:15.867237 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.867194    2549 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 17:41:15.867237 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.867224    2549 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 17:41:15.867237 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.867236    2549 kubelet.go:398] "Adding apiserver pod source"
Sep 10 17:41:15.867399 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.867246    2549 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 17:41:15.867618 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.867606    2549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 17:41:15.867647 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.867624    2549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 17:41:15.869709 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.869679    2549 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 17:41:15.871951 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.871934    2549 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 17:41:15.872018 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.871965    2549 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 17:41:15.872137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.872126    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 17:41:15.872167 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.872142    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 17:41:15.872167 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.872148    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 17:41:15.872167 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.872159    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 17:41:15.872254 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.872168    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 17:41:15.872254 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.872177    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 17:41:15.872678 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.872669    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 17:41:15.872678 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.872678    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 17:41:15.872758 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.872685    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 17:41:15.872758 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.872702    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 17:41:15.872758 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.872718    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 17:41:15.872758 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.872726    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 17:41:15.873958 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.873944    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 17:41:15.873958 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.873956    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 17:41:15.878766 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.878746    2549 server.go:1268] "Started kubelet"
Sep 10 17:41:15.878889 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.878822    2549 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 17:41:15.878889 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.878874    2549 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 17:41:15.879019 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.878920    2549 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 17:41:15.879140 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.879123    2549 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-245.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 17:41:15.879708 ip-10-0-139-245 systemd[1]: Started Kubernetes Kubelet.
Sep 10 17:41:15.879893 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:15.879793    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-245.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 17:41:15.879893 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:15.879808    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 17:41:15.880922 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.879954    2549 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 17:41:15.881542 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.881406    2549 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 17:41:15.883992 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.883968    2549 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 17:41:15.884077 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.883991    2549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 17:41:15.884193 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:15.884154    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-245.ec2.internal\" not found"
Sep 10 17:41:15.884810 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.884789    2549 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 17:41:15.884810 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.884809    2549 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 17:41:15.884945 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.884789    2549 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 17:41:15.885010 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.884947    2549 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 17:41:15.885010 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.884955    2549 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 17:41:15.885730 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:15.884821    2549 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-245.ec2.internal.18d4067aa9df9c46  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-245.ec2.internal,UID:ip-10-0-139-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-245.ec2.internal,},FirstTimestamp:2026-09-10 17:41:15.878710342 +0000 UTC m=+0.369762141,LastTimestamp:2026-09-10 17:41:15.878710342 +0000 UTC m=+0.369762141,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-245.ec2.internal,}"
Sep 10 17:41:15.885859 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:15.885840    2549 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 17:41:15.886787 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.886666    2549 factory.go:153] Registering CRI-O factory
Sep 10 17:41:15.886787 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.886730    2549 factory.go:223] Registration of the crio container factory successfully
Sep 10 17:41:15.886787 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.886781    2549 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 17:41:15.886902 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.886792    2549 factory.go:55] Registering systemd factory
Sep 10 17:41:15.886902 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.886801    2549 factory.go:223] Registration of the systemd container factory successfully
Sep 10 17:41:15.886902 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.886822    2549 factory.go:103] Registering Raw factory
Sep 10 17:41:15.886902 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.886836    2549 manager.go:1196] Started watching for new ooms in manager
Sep 10 17:41:15.887220 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.887207    2549 manager.go:349] Starting recovery of all containers
Sep 10 17:41:15.890621 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:15.890588    2549 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-245.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 17:41:15.890621 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:15.890603    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 17:41:15.897516 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.897484    2549 manager.go:354] Recovery completed
Sep 10 17:41:15.903649 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.903632    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:41:15.905642 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.905612    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:41:15.905814 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.905656    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:41:15.905814 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.905670    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:41:15.906354 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.906338    2549 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 17:41:15.906354 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.906354    2549 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 17:41:15.906485 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.906374    2549 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 17:41:15.906485 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.906342    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:41:15.907239 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.907220    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:41:15.907508 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.907333    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:41:15.907589 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.907520    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:41:15.907589 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:15.907374    2549 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-245.ec2.internal.18d4067aab7a872b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-245.ec2.internal,UID:ip-10-0-139-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-245.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-245.ec2.internal,},FirstTimestamp:2026-09-10 17:41:15.905640235 +0000 UTC m=+0.396692034,LastTimestamp:2026-09-10 17:41:15.905640235 +0000 UTC m=+0.396692034,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-245.ec2.internal,}"
Sep 10 17:41:15.908508 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.908495    2549 policy_none.go:49] "None policy: Start"
Sep 10 17:41:15.908575 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.908514    2549 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 17:41:15.908575 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.908528    2549 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 17:41:15.910474 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.910461    2549 policy_none.go:47] "Start"
Sep 10 17:41:15.916866 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:15.916797    2549 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-245.ec2.internal.18d4067aab7ae0f6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-245.ec2.internal,UID:ip-10-0-139-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-245.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-245.ec2.internal,},FirstTimestamp:2026-09-10 17:41:15.905663222 +0000 UTC m=+0.396715021,LastTimestamp:2026-09-10 17:41:15.905663222 +0000 UTC m=+0.396715021,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-245.ec2.internal,}"
Sep 10 17:41:15.929923 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:15.929847    2549 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-245.ec2.internal.18d4067aab7b0e0d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-245.ec2.internal,UID:ip-10-0-139-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-245.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-245.ec2.internal,},FirstTimestamp:2026-09-10 17:41:15.905674765 +0000 UTC m=+0.396726564,LastTimestamp:2026-09-10 17:41:15.905674765 +0000 UTC m=+0.396726564,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-245.ec2.internal,}"
Sep 10 17:41:15.932175 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.932156    2549 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6nk5f"
Sep 10 17:41:15.938589 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:15.938516    2549 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-139-245.ec2.internal.18d4067aab7a872b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-245.ec2.internal.18d4067aab7a872b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-245.ec2.internal,UID:ip-10-0-139-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-245.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-245.ec2.internal,},FirstTimestamp:2026-09-10 17:41:15.905640235 +0000 UTC m=+0.396692034,LastTimestamp:2026-09-10 17:41:15.907319624 +0000 UTC m=+0.398371427,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-245.ec2.internal,}"
Sep 10 17:41:15.964011 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:15.940428    2549 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6nk5f"
Sep 10 17:41:15.984748 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:15.984717    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-245.ec2.internal\" not found"
Sep 10 17:41:16.021465 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.021428    2549 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 17:41:16.022684 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.022664    2549 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 17:41:16.022799 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.022736    2549 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 17:41:16.022799 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.022761    2549 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 17:41:16.022911 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.022868    2549 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 10 17:41:16.024926 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.024907    2549 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:41:16.079330 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.079311    2549 manager.go:337] "Starting Device Plugin manager"
Sep 10 17:41:16.079435 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.079359    2549 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 17:41:16.079435 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.079369    2549 server.go:87] "Starting device plugin registration server"
Sep 10 17:41:16.079782 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.079770    2549 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 17:41:16.079824 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.079783    2549 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 17:41:16.079892 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.079878    2549 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 17:41:16.079974 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.079958    2549 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 17:41:16.079974 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.079970    2549 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 17:41:16.080565 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.080545    2549 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 17:41:16.080653 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.080593    2549 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-245.ec2.internal\" not found"
Sep 10 17:41:16.096486 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.096458    2549 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-245.ec2.internal\" not found" node="ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.123632 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.123598    2549 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-245.ec2.internal"]
Sep 10 17:41:16.123757 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.123687    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:41:16.125148 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.125131    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:41:16.125148 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.125159    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:41:16.125293 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.125170    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:41:16.127541 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.127529    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:41:16.127666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.127649    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.127723 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.127686    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:41:16.128331 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.128315    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:41:16.128397 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.128345    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:41:16.128397 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.128359    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:41:16.128397 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.128315    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:41:16.128397 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.128389    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:41:16.128558 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.128404    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:41:16.130544 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.130525    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.130653 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.130553    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:41:16.131227 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.131212    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:41:16.131305 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.131243    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:41:16.131305 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.131257    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:41:16.147337 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.147314    2549 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-245.ec2.internal\" not found" node="ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.150904 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.150888    2549 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-245.ec2.internal\" not found" node="ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.180395 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.180365    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:41:16.181335 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.181316    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:41:16.181442 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.181354    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:41:16.181442 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.181367    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:41:16.181442 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.181394    2549 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.186450 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.186427    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/269503f721ee6358f709c500ffc1e649-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal\" (UID: \"269503f721ee6358f709c500ffc1e649\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.186498 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.186463    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/71f8e2acfe6edbc5bafb0547bd10904c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-245.ec2.internal\" (UID: \"71f8e2acfe6edbc5bafb0547bd10904c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.186498 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.186488    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/269503f721ee6358f709c500ffc1e649-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal\" (UID: \"269503f721ee6358f709c500ffc1e649\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.187617 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.187606    2549 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.187661 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.187625    2549 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-245.ec2.internal\": node \"ip-10-0-139-245.ec2.internal\" not found"
Sep 10 17:41:16.204063 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.204042    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-245.ec2.internal\" not found"
Sep 10 17:41:16.287235 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.287199    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/269503f721ee6358f709c500ffc1e649-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal\" (UID: \"269503f721ee6358f709c500ffc1e649\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.287235 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.287235    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/71f8e2acfe6edbc5bafb0547bd10904c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-245.ec2.internal\" (UID: \"71f8e2acfe6edbc5bafb0547bd10904c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.287433 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.287254    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/269503f721ee6358f709c500ffc1e649-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal\" (UID: \"269503f721ee6358f709c500ffc1e649\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.287433 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.287297    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/269503f721ee6358f709c500ffc1e649-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal\" (UID: \"269503f721ee6358f709c500ffc1e649\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.287433 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.287316    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/269503f721ee6358f709c500ffc1e649-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal\" (UID: \"269503f721ee6358f709c500ffc1e649\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.287433 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.287316    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/71f8e2acfe6edbc5bafb0547bd10904c-config\") pod \"kube-apiserver-proxy-ip-10-0-139-245.ec2.internal\" (UID: \"71f8e2acfe6edbc5bafb0547bd10904c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.304337 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.304304    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-245.ec2.internal\" not found"
Sep 10 17:41:16.405271 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.405179    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-245.ec2.internal\" not found"
Sep 10 17:41:16.452537 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.452496    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.453733 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.453711    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-245.ec2.internal"
Sep 10 17:41:16.505595 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.505555    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-245.ec2.internal\" not found"
Sep 10 17:41:16.606816 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.606760    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-245.ec2.internal\" not found"
Sep 10 17:41:16.708174 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.708095    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-245.ec2.internal\" not found"
Sep 10 17:41:16.808482 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.808437    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-245.ec2.internal\" not found"
Sep 10 17:41:16.827796 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.827762    2549 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 17:41:16.827944 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.827924    2549 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 17:41:16.909259 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:16.909227    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-245.ec2.internal\" not found"
Sep 10 17:41:16.944973 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.944923    2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 17:36:16 +0000 UTC" deadline="2028-06-01 16:21:03.576573056 +0000 UTC"
Sep 10 17:41:16.944973 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:16.944976    2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15118h39m46.631602405s"
Sep 10 17:41:17.003446 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.003372    2549 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:41:17.010373 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:17.010351    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-245.ec2.internal\" not found"
Sep 10 17:41:17.029433 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.029412    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:41:17.030417 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.030402    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:41:17.030478 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.030437    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:41:17.030478 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.030453    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:41:17.058651 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:17.058619    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod269503f721ee6358f709c500ffc1e649.slice/crio-50fa1d60d62db4a5adf4ecfd749e605f4f83215cf7795f7d4110fb90208aba2a WatchSource:0}: Error finding container 50fa1d60d62db4a5adf4ecfd749e605f4f83215cf7795f7d4110fb90208aba2a: Status 404 returned error can't find the container with id 50fa1d60d62db4a5adf4ecfd749e605f4f83215cf7795f7d4110fb90208aba2a
Sep 10 17:41:17.058891 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:17.058870    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f8e2acfe6edbc5bafb0547bd10904c.slice/crio-bf35680a831693b914c55b1cb619fffc6a6cd42e3f326ce32a79351620ef0d0e WatchSource:0}: Error finding container bf35680a831693b914c55b1cb619fffc6a6cd42e3f326ce32a79351620ef0d0e: Status 404 returned error can't find the container with id bf35680a831693b914c55b1cb619fffc6a6cd42e3f326ce32a79351620ef0d0e
Sep 10 17:41:17.065528 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.065510    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:41:17.112084 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:17.112015    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-245.ec2.internal\" not found"
Sep 10 17:41:17.117886 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.117865    2549 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:41:17.188634 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.188601    2549 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-245.ec2.internal"
Sep 10 17:41:17.201807 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.201779    2549 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 17:41:17.202981 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.202965    2549 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal"
Sep 10 17:41:17.215773 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.215749    2549 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 17:41:17.304520 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.304429    2549 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:41:17.869254 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.869219    2549 apiserver.go:52] "Watching apiserver"
Sep 10 17:41:17.879156 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.879128    2549 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 17:41:17.879588 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.879551    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-jpdgj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal","openshift-multus/multus-additional-cni-plugins-9lfxv","openshift-multus/multus-r676t","openshift-multus/network-metrics-daemon-tnrvx","openshift-network-diagnostics/network-check-target-5q47l","kube-system/global-pull-secret-syncer-mj85b","kube-system/kube-apiserver-proxy-ip-10-0-139-245.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw","openshift-dns/node-resolver-kdng8","openshift-network-operator/iptables-alerter-67z82","openshift-ovn-kubernetes/ovnkube-node-krlkl","kube-system/konnectivity-agent-zbhnk","openshift-cluster-node-tuning-operator/tuned-5trnt"]
Sep 10 17:41:17.882429 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.882407    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:17.882551 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:17.882493    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:17.884567 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.884547    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zbhnk"
Sep 10 17:41:17.884698 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.884673    2549 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 17:41:17.886905 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.886883    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.888614 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.888594    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 17:41:17.888714 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.888622    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 17:41:17.888714 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.888595    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ksw5r\""
Sep 10 17:41:17.889285 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.889264    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.889913 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.889891    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 17:41:17.890832 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.890816    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 17:41:17.890931 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.890918    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 17:41:17.891387 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.891229    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 17:41:17.891387 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.891293    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 17:41:17.892063 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.892044    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:17.892155 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:17.892119    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:17.892874 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.892852    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cnfmh\""
Sep 10 17:41:17.892985 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.892942    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 17:41:17.893095 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.893079    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4tpvn\""
Sep 10 17:41:17.894288 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.894169    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:17.894288 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:17.894248    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:17.895177 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.895153    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/213a7f60-9ca9-48fc-824e-6c5c14451aad-cnibin\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.895244 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.895190    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbtqr\" (UniqueName: \"kubernetes.io/projected/213a7f60-9ca9-48fc-824e-6c5c14451aad-kube-api-access-jbtqr\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.895244 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.895215    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8cae49c8-331a-409b-8f6a-7d096c469ca5-dbus\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:17.895244 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.895240    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/81cddb05-3fd0-4ed3-83b6-e8d96dd71dcd-agent-certs\") pod \"konnectivity-agent-zbhnk\" (UID: \"81cddb05-3fd0-4ed3-83b6-e8d96dd71dcd\") " pod="kube-system/konnectivity-agent-zbhnk"
Sep 10 17:41:17.895410 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.895270    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/213a7f60-9ca9-48fc-824e-6c5c14451aad-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.895410 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.895297    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/213a7f60-9ca9-48fc-824e-6c5c14451aad-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.895410 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.895370    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/81cddb05-3fd0-4ed3-83b6-e8d96dd71dcd-konnectivity-ca\") pod \"konnectivity-agent-zbhnk\" (UID: \"81cddb05-3fd0-4ed3-83b6-e8d96dd71dcd\") " pod="kube-system/konnectivity-agent-zbhnk"
Sep 10 17:41:17.895568 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.895407    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/213a7f60-9ca9-48fc-824e-6c5c14451aad-cni-binary-copy\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.895568 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.895437    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/213a7f60-9ca9-48fc-824e-6c5c14451aad-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.895568 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.895463    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:17.895568 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.895490    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8cae49c8-331a-409b-8f6a-7d096c469ca5-kubelet-config\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:17.895568 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.895524    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/213a7f60-9ca9-48fc-824e-6c5c14451aad-system-cni-dir\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.896878 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.896855    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:17.898280 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.898255    2549 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 17:41:17.901684 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.900217    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jpdgj"
Sep 10 17:41:17.901684 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.900630    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kchp5\""
Sep 10 17:41:17.901684 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.901534    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 17:41:17.901919 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.901867    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 17:41:17.903186 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.902109    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 17:41:17.903853 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.903826    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 17:41:17.903958 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.903902    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8gbqd\""
Sep 10 17:41:17.904283 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.904265    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 17:41:17.904790 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.904752    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:17.905630 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.905074    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 17:41:17.908561 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.907911    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ptr64\""
Sep 10 17:41:17.908561 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.908102    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 17:41:17.908561 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.908264    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 17:41:17.908561 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.908273    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kdng8"
Sep 10 17:41:17.910376 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.910357    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-67z82"
Sep 10 17:41:17.910945 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.910925    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-47zt4\""
Sep 10 17:41:17.911460 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.911356    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 17:41:17.911460 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.911448    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 17:41:17.913213 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.913194    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.914097 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.914076    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 17:41:17.914195 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.914121    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 17:41:17.914297 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.914277    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qlqs7\""
Sep 10 17:41:17.914361 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.914307    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 17:41:17.916326 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.916306    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 17:41:17.916709 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.916680    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 17:41:17.919612 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.919412    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 17:41:17.919788 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.919743    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7pdqt\""
Sep 10 17:41:17.919922 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.919824    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 17:41:17.920148 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.920133    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 17:41:17.920240 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.920135    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 17:41:17.931208 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.931176    2549 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-g6prb"
Sep 10 17:41:17.947911 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.947886    2549 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-g6prb"
Sep 10 17:41:17.986779 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.986746    2549 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 17:41:17.995947 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.995915    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgzgq\" (UniqueName: \"kubernetes.io/projected/d3bb440a-577b-4df2-bb07-dd8ce1870717-kube-api-access-xgzgq\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.995947 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.995950    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-systemd\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:17.996166 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.995975    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqbxt\" (UniqueName: \"kubernetes.io/projected/50a3c219-25ae-42e8-9ed7-ae827178226a-kube-api-access-vqbxt\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.996166 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996007    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-tmp\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:17.996166 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996061    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwl8d\" (UniqueName: \"kubernetes.io/projected/df747e4e-bb42-420a-90e5-318f0b4f31e8-kube-api-access-nwl8d\") pod \"node-resolver-kdng8\" (UID: \"df747e4e-bb42-420a-90e5-318f0b4f31e8\") " pod="openshift-dns/node-resolver-kdng8"
Sep 10 17:41:17.996166 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996129    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-slash\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.996377 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996181    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-run-ovn\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.996377 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996209    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqb5s\" (UniqueName: \"kubernetes.io/projected/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-kube-api-access-xqb5s\") pod \"network-metrics-daemon-tnrvx\" (UID: \"78bdfa50-029c-4951-9ca4-4bfa31adf9a8\") " pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:17.996377 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996227    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-sys-fs\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:17.996377 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996247    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-multus-socket-dir-parent\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.996377 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996296    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-log-socket\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.996377 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996320    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/213a7f60-9ca9-48fc-824e-6c5c14451aad-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.996377 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996340    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3bb440a-577b-4df2-bb07-dd8ce1870717-multus-daemon-config\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.996377 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996375    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-modprobe-d\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:17.996800 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996493    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-sysconfig\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:17.996800 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996519    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-sysctl-conf\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:17.996800 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996595    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-run-netns\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.996800 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996633    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-run-ovn-kubernetes\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.996800 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996663    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-system-cni-dir\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.996800 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996701    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-var-lib-kubelet\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.996800 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996771    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-tuned\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:17.997101 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996805    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7d8x\" (UniqueName: \"kubernetes.io/projected/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-kube-api-access-r7d8x\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:17.997101 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996846    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-run-openvswitch\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.997101 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996875    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-node-log\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.997101 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996900    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-cnibin\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.997101 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996925    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-run-netns\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.997101 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996966    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/213a7f60-9ca9-48fc-824e-6c5c14451aad-cnibin\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.997101 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.996988    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-run\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:17.997101 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997013    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-run-systemd\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.997101 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997028    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/50a3c219-25ae-42e8-9ed7-ae827178226a-env-overrides\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.997101 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997048    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-run-k8s-cni-cncf-io\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.997101 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997029    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/213a7f60-9ca9-48fc-824e-6c5c14451aad-cnibin\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.997101 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997064    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t66dr\" (UniqueName: \"kubernetes.io/projected/1c14a9e5-fec4-4b25-b3a2-99d9bb6be063-kube-api-access-t66dr\") pod \"node-ca-jpdgj\" (UID: \"1c14a9e5-fec4-4b25-b3a2-99d9bb6be063\") " pod="openshift-image-registry/node-ca-jpdgj"
Sep 10 17:41:17.997101 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997082    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c013382-7f00-4396-802f-b54d938cff60-iptables-alerter-script\") pod \"iptables-alerter-67z82\" (UID: \"8c013382-7f00-4396-802f-b54d938cff60\") " pod="openshift-network-operator/iptables-alerter-67z82"
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997124    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997192    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-var-lib-cni-bin\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997231    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-kubernetes\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997255    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-sys\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:17.997290    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997302    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df747e4e-bb42-420a-90e5-318f0b4f31e8-tmp-dir\") pod \"node-resolver-kdng8\" (UID: \"df747e4e-bb42-420a-90e5-318f0b4f31e8\") " pod="openshift-dns/node-resolver-kdng8"
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997335    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-var-lib-openvswitch\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:17.997396    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret podName:8cae49c8-331a-409b-8f6a-7d096c469ca5 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:18.497347416 +0000 UTC m=+2.988399216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret") pod "global-pull-secret-syncer-mj85b" (UID: "8cae49c8-331a-409b-8f6a-7d096c469ca5") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997419    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs\") pod \"network-metrics-daemon-tnrvx\" (UID: \"78bdfa50-029c-4951-9ca4-4bfa31adf9a8\") " pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997451    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr97d\" (UniqueName: \"kubernetes.io/projected/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-kube-api-access-pr97d\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997479    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z955m\" (UniqueName: \"kubernetes.io/projected/8c013382-7f00-4396-802f-b54d938cff60-kube-api-access-z955m\") pod \"iptables-alerter-67z82\" (UID: \"8c013382-7f00-4396-802f-b54d938cff60\") " pod="openshift-network-operator/iptables-alerter-67z82"
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997501    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-etc-kubernetes\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997518    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-host\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997541    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/df747e4e-bb42-420a-90e5-318f0b4f31e8-hosts-file\") pod \"node-resolver-kdng8\" (UID: \"df747e4e-bb42-420a-90e5-318f0b4f31e8\") " pod="openshift-dns/node-resolver-kdng8"
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997564    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1c14a9e5-fec4-4b25-b3a2-99d9bb6be063-serviceca\") pod \"node-ca-jpdgj\" (UID: \"1c14a9e5-fec4-4b25-b3a2-99d9bb6be063\") " pod="openshift-image-registry/node-ca-jpdgj"
Sep 10 17:41:17.997695 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997603    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-multus-cni-dir\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997632    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-multus-conf-dir\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997669    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-etc-openvswitch\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997674    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/213a7f60-9ca9-48fc-824e-6c5c14451aad-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997700    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/50a3c219-25ae-42e8-9ed7-ae827178226a-ovn-node-metrics-cert\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997751    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c013382-7f00-4396-802f-b54d938cff60-host-slash\") pod \"iptables-alerter-67z82\" (UID: \"8c013382-7f00-4396-802f-b54d938cff60\") " pod="openshift-network-operator/iptables-alerter-67z82"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997785    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/81cddb05-3fd0-4ed3-83b6-e8d96dd71dcd-konnectivity-ca\") pod \"konnectivity-agent-zbhnk\" (UID: \"81cddb05-3fd0-4ed3-83b6-e8d96dd71dcd\") " pod="kube-system/konnectivity-agent-zbhnk"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997845    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-hostroot\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997868    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-var-lib-kubelet\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997887    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/50a3c219-25ae-42e8-9ed7-ae827178226a-ovnkube-script-lib\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997907    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-etc-selinux\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997932    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-systemd-units\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997959    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.997996    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-registration-dir\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998026    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-var-lib-cni-multus\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998065    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-kubelet\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.998386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998088    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-device-dir\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998107    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/213a7f60-9ca9-48fc-824e-6c5c14451aad-system-cni-dir\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998135    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbtqr\" (UniqueName: \"kubernetes.io/projected/213a7f60-9ca9-48fc-824e-6c5c14451aad-kube-api-access-jbtqr\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998182    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8cae49c8-331a-409b-8f6a-7d096c469ca5-dbus\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998220    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/81cddb05-3fd0-4ed3-83b6-e8d96dd71dcd-agent-certs\") pod \"konnectivity-agent-zbhnk\" (UID: \"81cddb05-3fd0-4ed3-83b6-e8d96dd71dcd\") " pod="kube-system/konnectivity-agent-zbhnk"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998252    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/213a7f60-9ca9-48fc-824e-6c5c14451aad-system-cni-dir\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998254    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-run-multus-certs\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998287    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8cae49c8-331a-409b-8f6a-7d096c469ca5-dbus\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998307    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/81cddb05-3fd0-4ed3-83b6-e8d96dd71dcd-konnectivity-ca\") pod \"konnectivity-agent-zbhnk\" (UID: \"81cddb05-3fd0-4ed3-83b6-e8d96dd71dcd\") " pod="kube-system/konnectivity-agent-zbhnk"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998402    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-cni-netd\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998446    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjtkq\" (UniqueName: \"kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq\") pod \"network-check-target-5q47l\" (UID: \"d61add86-4475-4f8f-b6d4-b3b298df34f4\") " pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998530    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998570    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3bb440a-577b-4df2-bb07-dd8ce1870717-cni-binary-copy\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998599    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/213a7f60-9ca9-48fc-824e-6c5c14451aad-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998657    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/213a7f60-9ca9-48fc-824e-6c5c14451aad-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998660    2549 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998693    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c14a9e5-fec4-4b25-b3a2-99d9bb6be063-host\") pod \"node-ca-jpdgj\" (UID: \"1c14a9e5-fec4-4b25-b3a2-99d9bb6be063\") " pod="openshift-image-registry/node-ca-jpdgj"
Sep 10 17:41:17.999137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998753    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-lib-modules\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.000010 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998799    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-cni-bin\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.000010 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998826    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/50a3c219-25ae-42e8-9ed7-ae827178226a-ovnkube-config\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.000010 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998854    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/213a7f60-9ca9-48fc-824e-6c5c14451aad-cni-binary-copy\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:18.000010 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.998868    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/213a7f60-9ca9-48fc-824e-6c5c14451aad-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:18.000010 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.999147    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-sysctl-d\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.000010 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.999181    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-socket-dir\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.000010 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.999212    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8cae49c8-331a-409b-8f6a-7d096c469ca5-kubelet-config\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:18.000010 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.999345    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8cae49c8-331a-409b-8f6a-7d096c469ca5-kubelet-config\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:18.000010 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.999346    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/213a7f60-9ca9-48fc-824e-6c5c14451aad-cni-binary-copy\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:18.000010 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:17.999381    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/213a7f60-9ca9-48fc-824e-6c5c14451aad-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:18.004498 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.004473    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/81cddb05-3fd0-4ed3-83b6-e8d96dd71dcd-agent-certs\") pod \"konnectivity-agent-zbhnk\" (UID: \"81cddb05-3fd0-4ed3-83b6-e8d96dd71dcd\") " pod="kube-system/konnectivity-agent-zbhnk"
Sep 10 17:41:18.009960 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.009931    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbtqr\" (UniqueName: \"kubernetes.io/projected/213a7f60-9ca9-48fc-824e-6c5c14451aad-kube-api-access-jbtqr\") pod \"multus-additional-cni-plugins-9lfxv\" (UID: \"213a7f60-9ca9-48fc-824e-6c5c14451aad\") " pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:18.029336 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.029267    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-245.ec2.internal" event={"ID":"71f8e2acfe6edbc5bafb0547bd10904c","Type":"ContainerStarted","Data":"bf35680a831693b914c55b1cb619fffc6a6cd42e3f326ce32a79351620ef0d0e"}
Sep 10 17:41:18.030893 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.030863    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal" event={"ID":"269503f721ee6358f709c500ffc1e649","Type":"ContainerStarted","Data":"50fa1d60d62db4a5adf4ecfd749e605f4f83215cf7795f7d4110fb90208aba2a"}
Sep 10 17:41:18.100069 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100029    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-systemd-units\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.100257 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100083    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.100257 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100106    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-systemd-units\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.100257 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100114    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-registration-dir\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.100257 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100168    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-registration-dir\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.100257 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100171    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-var-lib-cni-multus\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.100257 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100199    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-kubelet\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.100257 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100224    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-device-dir\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.100257 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100221    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100289    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-kubelet\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100293    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-device-dir\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100347    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-var-lib-cni-multus\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100384    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-run-multus-certs\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100411    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-cni-netd\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100438    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjtkq\" (UniqueName: \"kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq\") pod \"network-check-target-5q47l\" (UID: \"d61add86-4475-4f8f-b6d4-b3b298df34f4\") " pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100463    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100489    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3bb440a-577b-4df2-bb07-dd8ce1870717-cni-binary-copy\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100505    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-cni-netd\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100516    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c14a9e5-fec4-4b25-b3a2-99d9bb6be063-host\") pod \"node-ca-jpdgj\" (UID: \"1c14a9e5-fec4-4b25-b3a2-99d9bb6be063\") " pod="openshift-image-registry/node-ca-jpdgj"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100538    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100558    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-run-multus-certs\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100542    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-lib-modules\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100601    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c14a9e5-fec4-4b25-b3a2-99d9bb6be063-host\") pod \"node-ca-jpdgj\" (UID: \"1c14a9e5-fec4-4b25-b3a2-99d9bb6be063\") " pod="openshift-image-registry/node-ca-jpdgj"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100619    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-cni-bin\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100648    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/50a3c219-25ae-42e8-9ed7-ae827178226a-ovnkube-config\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.100666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100664    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-lib-modules\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100676    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-sysctl-d\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100707    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-cni-bin\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100704    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-socket-dir\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100747    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgzgq\" (UniqueName: \"kubernetes.io/projected/d3bb440a-577b-4df2-bb07-dd8ce1870717-kube-api-access-xgzgq\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100838    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-systemd\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100877    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqbxt\" (UniqueName: \"kubernetes.io/projected/50a3c219-25ae-42e8-9ed7-ae827178226a-kube-api-access-vqbxt\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100884    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-sysctl-d\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100892    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-socket-dir\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100938    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-systemd\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.100924    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-tmp\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101002    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwl8d\" (UniqueName: \"kubernetes.io/projected/df747e4e-bb42-420a-90e5-318f0b4f31e8-kube-api-access-nwl8d\") pod \"node-resolver-kdng8\" (UID: \"df747e4e-bb42-420a-90e5-318f0b4f31e8\") " pod="openshift-dns/node-resolver-kdng8"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101029    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-slash\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101053    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-run-ovn\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101085    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqb5s\" (UniqueName: \"kubernetes.io/projected/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-kube-api-access-xqb5s\") pod \"network-metrics-daemon-tnrvx\" (UID: \"78bdfa50-029c-4951-9ca4-4bfa31adf9a8\") " pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101110    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-sys-fs\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101116    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-slash\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.101492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101117    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-run-ovn\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101136    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-multus-socket-dir-parent\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101193    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3bb440a-577b-4df2-bb07-dd8ce1870717-cni-binary-copy\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101199    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-multus-socket-dir-parent\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101230    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-log-socket\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101245    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/50a3c219-25ae-42e8-9ed7-ae827178226a-ovnkube-config\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101258    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3bb440a-577b-4df2-bb07-dd8ce1870717-multus-daemon-config\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101282    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-sys-fs\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101285    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-modprobe-d\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101297    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-log-socket\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101320    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-sysconfig\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101346    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-sysctl-conf\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101358    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-sysconfig\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101375    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-run-netns\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101378    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-modprobe-d\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101404    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-run-ovn-kubernetes\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101421    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-run-netns\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101426    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-system-cni-dir\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.102476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101456    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-host-run-ovn-kubernetes\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101474    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-system-cni-dir\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101483    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-sysctl-conf\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101493    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-var-lib-kubelet\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101524    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-tuned\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101549    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7d8x\" (UniqueName: \"kubernetes.io/projected/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-kube-api-access-r7d8x\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101575    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-run-openvswitch\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101601    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-node-log\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101627    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-cnibin\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101634    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-run-openvswitch\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101599    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-var-lib-kubelet\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101650    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-run-netns\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101694    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-run\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101720    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-run-systemd\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101746    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/50a3c219-25ae-42e8-9ed7-ae827178226a-env-overrides\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101755    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-cnibin\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101695    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-node-log\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101788    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-run-k8s-cni-cncf-io\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.103423 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101808    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-run-netns\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101816    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t66dr\" (UniqueName: \"kubernetes.io/projected/1c14a9e5-fec4-4b25-b3a2-99d9bb6be063-kube-api-access-t66dr\") pod \"node-ca-jpdgj\" (UID: \"1c14a9e5-fec4-4b25-b3a2-99d9bb6be063\") " pod="openshift-image-registry/node-ca-jpdgj"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101820    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3bb440a-577b-4df2-bb07-dd8ce1870717-multus-daemon-config\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101833    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-run\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101843    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c013382-7f00-4396-802f-b54d938cff60-iptables-alerter-script\") pod \"iptables-alerter-67z82\" (UID: \"8c013382-7f00-4396-802f-b54d938cff60\") " pod="openshift-network-operator/iptables-alerter-67z82"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101895    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-run-k8s-cni-cncf-io\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.101908    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-run-systemd\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102036    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-var-lib-cni-bin\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102064    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-kubernetes\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102087    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-sys\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102114    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df747e4e-bb42-420a-90e5-318f0b4f31e8-tmp-dir\") pod \"node-resolver-kdng8\" (UID: \"df747e4e-bb42-420a-90e5-318f0b4f31e8\") " pod="openshift-dns/node-resolver-kdng8"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102140    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-var-lib-openvswitch\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102169    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs\") pod \"network-metrics-daemon-tnrvx\" (UID: \"78bdfa50-029c-4951-9ca4-4bfa31adf9a8\") " pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102196    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr97d\" (UniqueName: \"kubernetes.io/projected/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-kube-api-access-pr97d\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102224    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z955m\" (UniqueName: \"kubernetes.io/projected/8c013382-7f00-4396-802f-b54d938cff60-kube-api-access-z955m\") pod \"iptables-alerter-67z82\" (UID: \"8c013382-7f00-4396-802f-b54d938cff60\") " pod="openshift-network-operator/iptables-alerter-67z82"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102220    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/50a3c219-25ae-42e8-9ed7-ae827178226a-env-overrides\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102288    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-var-lib-openvswitch\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102358    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-kubernetes\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.104422 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:18.102373    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102387    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c013382-7f00-4396-802f-b54d938cff60-iptables-alerter-script\") pod \"iptables-alerter-67z82\" (UID: \"8c013382-7f00-4396-802f-b54d938cff60\") " pod="openshift-network-operator/iptables-alerter-67z82"
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102415    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-host-var-lib-cni-bin\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:18.102440    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs podName:78bdfa50-029c-4951-9ca4-4bfa31adf9a8 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:18.60242183 +0000 UTC m=+3.093473632 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs") pod "network-metrics-daemon-tnrvx" (UID: "78bdfa50-029c-4951-9ca4-4bfa31adf9a8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102482    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df747e4e-bb42-420a-90e5-318f0b4f31e8-tmp-dir\") pod \"node-resolver-kdng8\" (UID: \"df747e4e-bb42-420a-90e5-318f0b4f31e8\") " pod="openshift-dns/node-resolver-kdng8"
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102536    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-etc-kubernetes\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102563    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-host\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102573    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-sys\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102590    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/df747e4e-bb42-420a-90e5-318f0b4f31e8-hosts-file\") pod \"node-resolver-kdng8\" (UID: \"df747e4e-bb42-420a-90e5-318f0b4f31e8\") " pod="openshift-dns/node-resolver-kdng8"
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102614    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1c14a9e5-fec4-4b25-b3a2-99d9bb6be063-serviceca\") pod \"node-ca-jpdgj\" (UID: \"1c14a9e5-fec4-4b25-b3a2-99d9bb6be063\") " pod="openshift-image-registry/node-ca-jpdgj"
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102621    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-etc-kubernetes\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102649    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-multus-cni-dir\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102668    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-host\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102679    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-multus-conf-dir\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102694    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-multus-cni-dir\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102712    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/df747e4e-bb42-420a-90e5-318f0b4f31e8-hosts-file\") pod \"node-resolver-kdng8\" (UID: \"df747e4e-bb42-420a-90e5-318f0b4f31e8\") " pod="openshift-dns/node-resolver-kdng8"
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102716    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-etc-openvswitch\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.105363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102746    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-multus-conf-dir\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102755    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/50a3c219-25ae-42e8-9ed7-ae827178226a-ovn-node-metrics-cert\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102754    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/50a3c219-25ae-42e8-9ed7-ae827178226a-etc-openvswitch\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102797    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c013382-7f00-4396-802f-b54d938cff60-host-slash\") pod \"iptables-alerter-67z82\" (UID: \"8c013382-7f00-4396-802f-b54d938cff60\") " pod="openshift-network-operator/iptables-alerter-67z82"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102825    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-hostroot\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102837    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c013382-7f00-4396-802f-b54d938cff60-host-slash\") pod \"iptables-alerter-67z82\" (UID: \"8c013382-7f00-4396-802f-b54d938cff60\") " pod="openshift-network-operator/iptables-alerter-67z82"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102849    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-var-lib-kubelet\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102875    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/50a3c219-25ae-42e8-9ed7-ae827178226a-ovnkube-script-lib\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102901    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-etc-selinux\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102935    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-var-lib-kubelet\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102878    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3bb440a-577b-4df2-bb07-dd8ce1870717-hostroot\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.102977    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-etc-selinux\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.103272    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1c14a9e5-fec4-4b25-b3a2-99d9bb6be063-serviceca\") pod \"node-ca-jpdgj\" (UID: \"1c14a9e5-fec4-4b25-b3a2-99d9bb6be063\") " pod="openshift-image-registry/node-ca-jpdgj"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.103438    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/50a3c219-25ae-42e8-9ed7-ae827178226a-ovnkube-script-lib\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.104343    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-etc-tuned\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.105411    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/50a3c219-25ae-42e8-9ed7-ae827178226a-ovn-node-metrics-cert\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.106060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.105677    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-tmp\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.108247 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:18.108204    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:41:18.108247 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:18.108229    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:41:18.108247 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:18.108247    2549 projected.go:196] Error preparing data for projected volume kube-api-access-pjtkq for pod openshift-network-diagnostics/network-check-target-5q47l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:41:18.108513 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:18.108315    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq podName:d61add86-4475-4f8f-b6d4-b3b298df34f4 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:18.60829694 +0000 UTC m=+3.099348749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pjtkq" (UniqueName: "kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq") pod "network-check-target-5q47l" (UID: "d61add86-4475-4f8f-b6d4-b3b298df34f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:41:18.110862 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.110407    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwl8d\" (UniqueName: \"kubernetes.io/projected/df747e4e-bb42-420a-90e5-318f0b4f31e8-kube-api-access-nwl8d\") pod \"node-resolver-kdng8\" (UID: \"df747e4e-bb42-420a-90e5-318f0b4f31e8\") " pod="openshift-dns/node-resolver-kdng8"
Sep 10 17:41:18.111502 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.111430    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgzgq\" (UniqueName: \"kubernetes.io/projected/d3bb440a-577b-4df2-bb07-dd8ce1870717-kube-api-access-xgzgq\") pod \"multus-r676t\" (UID: \"d3bb440a-577b-4df2-bb07-dd8ce1870717\") " pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.111502 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.111435    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqb5s\" (UniqueName: \"kubernetes.io/projected/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-kube-api-access-xqb5s\") pod \"network-metrics-daemon-tnrvx\" (UID: \"78bdfa50-029c-4951-9ca4-4bfa31adf9a8\") " pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:18.111876 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.111851    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7d8x\" (UniqueName: \"kubernetes.io/projected/c9fa653b-727e-4a19-8b6a-b9ab6c332f77-kube-api-access-r7d8x\") pod \"tuned-5trnt\" (UID: \"c9fa653b-727e-4a19-8b6a-b9ab6c332f77\") " pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.111972 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.111922    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqbxt\" (UniqueName: \"kubernetes.io/projected/50a3c219-25ae-42e8-9ed7-ae827178226a-kube-api-access-vqbxt\") pod \"ovnkube-node-krlkl\" (UID: \"50a3c219-25ae-42e8-9ed7-ae827178226a\") " pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.113444 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.113367    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr97d\" (UniqueName: \"kubernetes.io/projected/9d16b6ee-5c09-4425-b3c2-fa4ee5366d63-kube-api-access-pr97d\") pod \"aws-ebs-csi-driver-node-rhbhw\" (UID: \"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.113780 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.113673    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t66dr\" (UniqueName: \"kubernetes.io/projected/1c14a9e5-fec4-4b25-b3a2-99d9bb6be063-kube-api-access-t66dr\") pod \"node-ca-jpdgj\" (UID: \"1c14a9e5-fec4-4b25-b3a2-99d9bb6be063\") " pod="openshift-image-registry/node-ca-jpdgj"
Sep 10 17:41:18.114204 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.114183    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z955m\" (UniqueName: \"kubernetes.io/projected/8c013382-7f00-4396-802f-b54d938cff60-kube-api-access-z955m\") pod \"iptables-alerter-67z82\" (UID: \"8c013382-7f00-4396-802f-b54d938cff60\") " pod="openshift-network-operator/iptables-alerter-67z82"
Sep 10 17:41:18.140592 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.140519    2549 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:41:18.198913 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.198876    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zbhnk"
Sep 10 17:41:18.209755 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.209726    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9lfxv"
Sep 10 17:41:18.218757 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.218736    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r676t"
Sep 10 17:41:18.226699 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.226671    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw"
Sep 10 17:41:18.234817 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.234795    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jpdgj"
Sep 10 17:41:18.242932 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.242909    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5trnt"
Sep 10 17:41:18.250855 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.250827    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kdng8"
Sep 10 17:41:18.260521 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.260491    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-67z82"
Sep 10 17:41:18.267382 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.267359    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:18.505815 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.505717    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:18.505992 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:18.505902    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:41:18.505992 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:18.505977    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret podName:8cae49c8-331a-409b-8f6a-7d096c469ca5 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:19.505960243 +0000 UTC m=+3.997012037 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret") pod "global-pull-secret-syncer-mj85b" (UID: "8cae49c8-331a-409b-8f6a-7d096c469ca5") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:41:18.606988 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.606947    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs\") pod \"network-metrics-daemon-tnrvx\" (UID: \"78bdfa50-029c-4951-9ca4-4bfa31adf9a8\") " pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:18.607180 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:18.607124    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:41:18.607244 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:18.607194    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs podName:78bdfa50-029c-4951-9ca4-4bfa31adf9a8 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:19.607177998 +0000 UTC m=+4.098229786 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs") pod "network-metrics-daemon-tnrvx" (UID: "78bdfa50-029c-4951-9ca4-4bfa31adf9a8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:41:18.708028 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.707988    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjtkq\" (UniqueName: \"kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq\") pod \"network-check-target-5q47l\" (UID: \"d61add86-4475-4f8f-b6d4-b3b298df34f4\") " pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:18.708228 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:18.708202    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:41:18.708283 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:18.708237    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:41:18.708283 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:18.708255    2549 projected.go:196] Error preparing data for projected volume kube-api-access-pjtkq for pod openshift-network-diagnostics/network-check-target-5q47l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:41:18.708359 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:18.708329    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq podName:d61add86-4475-4f8f-b6d4-b3b298df34f4 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:19.70830937 +0000 UTC m=+4.199361173 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pjtkq" (UniqueName: "kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq") pod "network-check-target-5q47l" (UID: "d61add86-4475-4f8f-b6d4-b3b298df34f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:41:18.952675 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.952542    2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 17:36:18 +0000 UTC" deadline="2028-04-24 01:33:44.101535427 +0000 UTC"
Sep 10 17:41:18.952675 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:18.952609    2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14191h52m25.148931358s"
Sep 10 17:41:19.136870 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:19.136631    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d16b6ee_5c09_4425_b3c2_fa4ee5366d63.slice/crio-ae80d7ab6be811185c82c5e2727923a5dfe19651f9d736f58b44f1013e7af03e WatchSource:0}: Error finding container ae80d7ab6be811185c82c5e2727923a5dfe19651f9d736f58b44f1013e7af03e: Status 404 returned error can't find the container with id ae80d7ab6be811185c82c5e2727923a5dfe19651f9d736f58b44f1013e7af03e
Sep 10 17:41:19.140847 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:19.140680    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c14a9e5_fec4_4b25_b3a2_99d9bb6be063.slice/crio-7a0b30ed4f4c000701d6d5a131871b294eb4dc286d0a9addd76af3a3820fb971 WatchSource:0}: Error finding container 7a0b30ed4f4c000701d6d5a131871b294eb4dc286d0a9addd76af3a3820fb971: Status 404 returned error can't find the container with id 7a0b30ed4f4c000701d6d5a131871b294eb4dc286d0a9addd76af3a3820fb971
Sep 10 17:41:19.515433 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:19.515105    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:19.515632 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:19.515265    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:41:19.515632 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:19.515558    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret podName:8cae49c8-331a-409b-8f6a-7d096c469ca5 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:21.515527219 +0000 UTC m=+6.006579028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret") pod "global-pull-secret-syncer-mj85b" (UID: "8cae49c8-331a-409b-8f6a-7d096c469ca5") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:41:19.616719 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:19.616682    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs\") pod \"network-metrics-daemon-tnrvx\" (UID: \"78bdfa50-029c-4951-9ca4-4bfa31adf9a8\") " pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:19.616927 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:19.616861    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:41:19.616996 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:19.616945    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs podName:78bdfa50-029c-4951-9ca4-4bfa31adf9a8 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:21.616920826 +0000 UTC m=+6.107972614 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs") pod "network-metrics-daemon-tnrvx" (UID: "78bdfa50-029c-4951-9ca4-4bfa31adf9a8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:41:19.717783 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:19.717691    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjtkq\" (UniqueName: \"kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq\") pod \"network-check-target-5q47l\" (UID: \"d61add86-4475-4f8f-b6d4-b3b298df34f4\") " pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:19.717960 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:19.717924    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:41:19.717960 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:19.717950    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:41:19.718096 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:19.717964    2549 projected.go:196] Error preparing data for projected volume kube-api-access-pjtkq for pod openshift-network-diagnostics/network-check-target-5q47l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:41:19.718096 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:19.718025    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq podName:d61add86-4475-4f8f-b6d4-b3b298df34f4 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:21.71800617 +0000 UTC m=+6.209057979 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pjtkq" (UniqueName: "kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq") pod "network-check-target-5q47l" (UID: "d61add86-4475-4f8f-b6d4-b3b298df34f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:41:19.954350 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:19.953746    2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 17:36:18 +0000 UTC" deadline="2028-04-14 12:09:13.025690977 +0000 UTC"
Sep 10 17:41:19.954350 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:19.953785    2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13962h27m53.07191009s"
Sep 10 17:41:20.026069 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.026031    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:20.026279 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:20.026190    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:20.026629 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.026606    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:20.026813 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:20.026789    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:20.026891 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.026864    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:20.026956 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:20.026939    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:20.050280 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.050234    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-245.ec2.internal" event={"ID":"71f8e2acfe6edbc5bafb0547bd10904c","Type":"ContainerStarted","Data":"da6bcb72cb718442214a531ad78789a3f28bb6bb4ccfcf17efd48e5a788ef926"}
Sep 10 17:41:20.059129 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.059091    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zbhnk" event={"ID":"81cddb05-3fd0-4ed3-83b6-e8d96dd71dcd","Type":"ContainerStarted","Data":"e1aae5c8cf9848bd9d63b03e25122815e9de75057c51e268635f91f7912381a8"}
Sep 10 17:41:20.063994 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.063959    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5trnt" event={"ID":"c9fa653b-727e-4a19-8b6a-b9ab6c332f77","Type":"ContainerStarted","Data":"605ed5827690cb828e080288fcc78a13e175757860e69db0bbcf4fe4ad6508a9"}
Sep 10 17:41:20.091181 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.087934    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r676t" event={"ID":"d3bb440a-577b-4df2-bb07-dd8ce1870717","Type":"ContainerStarted","Data":"55a52bc9b1e24051822198d1f74da91b5ca0248b70877832654e5a5c42a142f4"}
Sep 10 17:41:20.101571 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.101531    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw" event={"ID":"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63","Type":"ContainerStarted","Data":"ae80d7ab6be811185c82c5e2727923a5dfe19651f9d736f58b44f1013e7af03e"}
Sep 10 17:41:20.108893 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.108861    2549 generic.go:356] "Generic (PLEG): container finished" podID="269503f721ee6358f709c500ffc1e649" containerID="24e9613f6f5a647a3cfe6c51f360fa7a7c98d699618dbc8118942d0e785cae70" exitCode=0
Sep 10 17:41:20.109180 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.109155    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal" event={"ID":"269503f721ee6358f709c500ffc1e649","Type":"ContainerDied","Data":"24e9613f6f5a647a3cfe6c51f360fa7a7c98d699618dbc8118942d0e785cae70"}
Sep 10 17:41:20.122988 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.122960    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl" event={"ID":"50a3c219-25ae-42e8-9ed7-ae827178226a","Type":"ContainerStarted","Data":"6dfebee2efb92543e928132712311b4201707e6b124ff20a3d20642e063611b9"}
Sep 10 17:41:20.132528 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.132462    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-245.ec2.internal" podStartSLOduration=3.132443395 podStartE2EDuration="3.132443395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:20.074537157 +0000 UTC m=+4.565588947" watchObservedRunningTime="2026-09-10 17:41:20.132443395 +0000 UTC m=+4.623495208"
Sep 10 17:41:20.137842 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.137813    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lfxv" event={"ID":"213a7f60-9ca9-48fc-824e-6c5c14451aad","Type":"ContainerStarted","Data":"bca36a94570902c666afa09190f64bae211b207b633d401aa97442277ddee46e"}
Sep 10 17:41:20.153186 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.153083    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kdng8" event={"ID":"df747e4e-bb42-420a-90e5-318f0b4f31e8","Type":"ContainerStarted","Data":"2963d5d5d1c875a38d513e954320b6e4c9384a9469f1aea7720b33e73a6f7357"}
Sep 10 17:41:20.166866 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.166833    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-67z82" event={"ID":"8c013382-7f00-4396-802f-b54d938cff60","Type":"ContainerStarted","Data":"7ad10883db1ceae7753b2b1b923d18466c97d2b9f74bfeaab2a4fff4feaaded6"}
Sep 10 17:41:20.178251 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:20.178215    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jpdgj" event={"ID":"1c14a9e5-fec4-4b25-b3a2-99d9bb6be063","Type":"ContainerStarted","Data":"7a0b30ed4f4c000701d6d5a131871b294eb4dc286d0a9addd76af3a3820fb971"}
Sep 10 17:41:21.204109 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:21.204070    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal" event={"ID":"269503f721ee6358f709c500ffc1e649","Type":"ContainerStarted","Data":"cb1a3e60654624375032831e226a6d9a0bf610ae8fba60b22f54c5636c9a51d7"}
Sep 10 17:41:21.533943 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:21.533849    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:21.534119 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:21.534065    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:41:21.534203 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:21.534135    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret podName:8cae49c8-331a-409b-8f6a-7d096c469ca5 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:25.534114667 +0000 UTC m=+10.025166458 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret") pod "global-pull-secret-syncer-mj85b" (UID: "8cae49c8-331a-409b-8f6a-7d096c469ca5") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:41:21.635159 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:21.635118    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs\") pod \"network-metrics-daemon-tnrvx\" (UID: \"78bdfa50-029c-4951-9ca4-4bfa31adf9a8\") " pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:21.635437 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:21.635304    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:41:21.635437 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:21.635382    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs podName:78bdfa50-029c-4951-9ca4-4bfa31adf9a8 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:25.635361041 +0000 UTC m=+10.126412845 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs") pod "network-metrics-daemon-tnrvx" (UID: "78bdfa50-029c-4951-9ca4-4bfa31adf9a8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:41:21.736131 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:21.736089    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjtkq\" (UniqueName: \"kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq\") pod \"network-check-target-5q47l\" (UID: \"d61add86-4475-4f8f-b6d4-b3b298df34f4\") " pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:21.736330 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:21.736308    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:41:21.736424 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:21.736337    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:41:21.736424 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:21.736350    2549 projected.go:196] Error preparing data for projected volume kube-api-access-pjtkq for pod openshift-network-diagnostics/network-check-target-5q47l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:41:21.736424 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:21.736409    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq podName:d61add86-4475-4f8f-b6d4-b3b298df34f4 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:25.736389333 +0000 UTC m=+10.227441140 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pjtkq" (UniqueName: "kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq") pod "network-check-target-5q47l" (UID: "d61add86-4475-4f8f-b6d4-b3b298df34f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:41:22.025840 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:22.025419    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:22.025840 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:22.025457    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:22.025840 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:22.025457    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:22.025840 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:22.025583    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:22.025840 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:22.025676    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:22.025840 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:22.025794    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:24.025602 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:24.024896    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:24.025602 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:24.025049    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:24.025602 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:24.025466    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:24.025602 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:24.025552    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:24.026525 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:24.026365    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:24.026525 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:24.026468    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:25.566853 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:25.566809    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:25.567277 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:25.567063    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:41:25.567277 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:25.567132    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret podName:8cae49c8-331a-409b-8f6a-7d096c469ca5 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:33.567111959 +0000 UTC m=+18.058163764 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret") pod "global-pull-secret-syncer-mj85b" (UID: "8cae49c8-331a-409b-8f6a-7d096c469ca5") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:41:25.667874 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:25.667838    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs\") pod \"network-metrics-daemon-tnrvx\" (UID: \"78bdfa50-029c-4951-9ca4-4bfa31adf9a8\") " pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:25.668049 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:25.667945    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:41:25.668049 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:25.668007    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs podName:78bdfa50-029c-4951-9ca4-4bfa31adf9a8 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:33.667988737 +0000 UTC m=+18.159040536 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs") pod "network-metrics-daemon-tnrvx" (UID: "78bdfa50-029c-4951-9ca4-4bfa31adf9a8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:41:25.769076 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:25.769028    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjtkq\" (UniqueName: \"kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq\") pod \"network-check-target-5q47l\" (UID: \"d61add86-4475-4f8f-b6d4-b3b298df34f4\") " pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:25.769258 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:25.769200    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:41:25.769258 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:25.769224    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:41:25.769258 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:25.769236    2549 projected.go:196] Error preparing data for projected volume kube-api-access-pjtkq for pod openshift-network-diagnostics/network-check-target-5q47l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:41:25.769408 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:25.769296    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq podName:d61add86-4475-4f8f-b6d4-b3b298df34f4 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:33.769277885 +0000 UTC m=+18.260329688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pjtkq" (UniqueName: "kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq") pod "network-check-target-5q47l" (UID: "d61add86-4475-4f8f-b6d4-b3b298df34f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:41:26.024015 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:26.023377    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:26.024015 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:26.023409    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:26.024015 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:26.023505    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:26.024015 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:26.023377    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:26.024015 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:26.023930    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:26.024953 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:26.024907    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:28.023558 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:28.023528    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:28.024053 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:28.023670    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:28.024370 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:28.024158    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:28.024370 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:28.024251    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:28.024370 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:28.023472    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:28.024370 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:28.024341    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:30.023817 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:30.023748    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:30.023817 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:30.023769    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:30.024307 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:30.023748    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:30.024307 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:30.023948    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:30.024307 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:30.024071    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:30.024307 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:30.024135    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:32.023606 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:32.023569    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:32.024061 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:32.023569    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:32.024061 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:32.023737    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:32.024061 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:32.023569    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:32.024061 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:32.023813    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:32.024061 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:32.023879    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:33.627913 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:33.627616    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:33.627913 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:33.627791    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:41:33.628375 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:33.627987    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret podName:8cae49c8-331a-409b-8f6a-7d096c469ca5 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:49.627966522 +0000 UTC m=+34.119018325 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret") pod "global-pull-secret-syncer-mj85b" (UID: "8cae49c8-331a-409b-8f6a-7d096c469ca5") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:41:33.728947 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:33.728914    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs\") pod \"network-metrics-daemon-tnrvx\" (UID: \"78bdfa50-029c-4951-9ca4-4bfa31adf9a8\") " pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:33.729117 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:33.729045    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:41:33.729117 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:33.729102    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs podName:78bdfa50-029c-4951-9ca4-4bfa31adf9a8 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:49.729085352 +0000 UTC m=+34.220137142 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs") pod "network-metrics-daemon-tnrvx" (UID: "78bdfa50-029c-4951-9ca4-4bfa31adf9a8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:41:33.830109 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:33.830061    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjtkq\" (UniqueName: \"kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq\") pod \"network-check-target-5q47l\" (UID: \"d61add86-4475-4f8f-b6d4-b3b298df34f4\") " pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:33.830289 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:33.830235    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:41:33.830289 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:33.830252    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:41:33.830289 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:33.830262    2549 projected.go:196] Error preparing data for projected volume kube-api-access-pjtkq for pod openshift-network-diagnostics/network-check-target-5q47l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:41:33.830433 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:33.830315    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq podName:d61add86-4475-4f8f-b6d4-b3b298df34f4 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:49.83029751 +0000 UTC m=+34.321349310 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pjtkq" (UniqueName: "kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq") pod "network-check-target-5q47l" (UID: "d61add86-4475-4f8f-b6d4-b3b298df34f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:41:34.023126 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:34.023090    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:34.023315 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:34.023090    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:34.023315 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:34.023231    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:34.023315 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:34.023090    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:34.023471 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:34.023321    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:34.023471 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:34.023403    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:36.024669 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:36.024626    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:36.024669 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:36.024672    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:36.025198 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:36.024779    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:36.025198 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:36.024834    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:36.025198 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:36.024866    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:36.025198 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:36.024953    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:38.023899 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.023707    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:38.024614 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.023711    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:38.024614 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:38.023989    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:38.024614 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.023718    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:38.024614 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:38.024045    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:38.024614 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:38.024137    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:38.242621 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.242582    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl" event={"ID":"50a3c219-25ae-42e8-9ed7-ae827178226a","Type":"ContainerStarted","Data":"07918d07fd3dfdf250f5775629b533e9f5204c6d3e14755364eafb39cd651963"}
Sep 10 17:41:38.242857 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.242653    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl" event={"ID":"50a3c219-25ae-42e8-9ed7-ae827178226a","Type":"ContainerStarted","Data":"5e31f900368a159693145fc1366c8e75912cf639617e7be2f57bef109439df2a"}
Sep 10 17:41:38.242857 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.242718    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl" event={"ID":"50a3c219-25ae-42e8-9ed7-ae827178226a","Type":"ContainerStarted","Data":"8238ee9c248ccfb617473af509aab33bd240e21c75ee9048c5ff089f9c08ecf1"}
Sep 10 17:41:38.242857 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.242748    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl" event={"ID":"50a3c219-25ae-42e8-9ed7-ae827178226a","Type":"ContainerStarted","Data":"c45fdfaaba891aee8cce134e42631be23de511cc83d51f09b6893a20f27715bd"}
Sep 10 17:41:38.242857 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.242777    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl" event={"ID":"50a3c219-25ae-42e8-9ed7-ae827178226a","Type":"ContainerStarted","Data":"861db68d171a84d1d3eadfd0f65a280ba716e4504586e6371d1e8790661b5b8e"}
Sep 10 17:41:38.242857 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.242831    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl" event={"ID":"50a3c219-25ae-42e8-9ed7-ae827178226a","Type":"ContainerStarted","Data":"38cd72814e6beecd6a8fb20756cd7912beb6ee99b1f06bb3192081580fd1c37b"}
Sep 10 17:41:38.244078 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.244056    2549 generic.go:356] "Generic (PLEG): container finished" podID="213a7f60-9ca9-48fc-824e-6c5c14451aad" containerID="686153b2cf1a649e05d60217cb4f8f23383e2821d2b28c66a45ac04ef6f061c1" exitCode=0
Sep 10 17:41:38.244176 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.244149    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lfxv" event={"ID":"213a7f60-9ca9-48fc-824e-6c5c14451aad","Type":"ContainerDied","Data":"686153b2cf1a649e05d60217cb4f8f23383e2821d2b28c66a45ac04ef6f061c1"}
Sep 10 17:41:38.245452 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.245434    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kdng8" event={"ID":"df747e4e-bb42-420a-90e5-318f0b4f31e8","Type":"ContainerStarted","Data":"6b9b43b35387fc3590836ae70b03e605114f3379202cd9065bb756ad9d52f375"}
Sep 10 17:41:38.246706 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.246672    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jpdgj" event={"ID":"1c14a9e5-fec4-4b25-b3a2-99d9bb6be063","Type":"ContainerStarted","Data":"001250a1ee1ff43d6672118ed847bab47a6c58fe3d9a95611eced0e24689a04b"}
Sep 10 17:41:38.247870 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.247853    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zbhnk" event={"ID":"81cddb05-3fd0-4ed3-83b6-e8d96dd71dcd","Type":"ContainerStarted","Data":"f13a5676590f75fb6d2377645b28fe8ae93168cf3ddc16342d3bd278ea71378a"}
Sep 10 17:41:38.249077 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.249060    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5trnt" event={"ID":"c9fa653b-727e-4a19-8b6a-b9ab6c332f77","Type":"ContainerStarted","Data":"bc9c196176074a89a548a888811ece76d184166b2038e40d7aaf2acd75080af4"}
Sep 10 17:41:38.250392 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.250375    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r676t" event={"ID":"d3bb440a-577b-4df2-bb07-dd8ce1870717","Type":"ContainerStarted","Data":"200c26fd1aa34b4d57cad03a61a9376c1f15c06e598a62a7cbf0b764dd6a8bee"}
Sep 10 17:41:38.251754 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.251735    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw" event={"ID":"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63","Type":"ContainerStarted","Data":"dbad361b13a55703bd19e290624faab474fa0c0f6a1b6230d7a9c782cbed953c"}
Sep 10 17:41:38.269078 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.269037    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-245.ec2.internal" podStartSLOduration=21.269024089 podStartE2EDuration="21.269024089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:21.232467598 +0000 UTC m=+5.723519410" watchObservedRunningTime="2026-09-10 17:41:38.269024089 +0000 UTC m=+22.760075899"
Sep 10 17:41:38.308963 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.308890    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zbhnk" podStartSLOduration=4.376484687 podStartE2EDuration="22.308875225s" totalImagesPullingTime="17.932390538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:38.287362421 +0000 UTC m=+22.778414228" watchObservedRunningTime="2026-09-10 17:41:38.308875225 +0000 UTC m=+22.799927032"
Sep 10 17:41:38.309129 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.309109    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-r676t" podStartSLOduration=4.297037649 podStartE2EDuration="22.309104064s" totalImagesPullingTime="18.012066415s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:38.308668618 +0000 UTC m=+22.799720426" watchObservedRunningTime="2026-09-10 17:41:38.309104064 +0000 UTC m=+22.800155872"
Sep 10 17:41:38.354824 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.353599    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jpdgj" podStartSLOduration=4.417079711 podStartE2EDuration="22.353579472s" totalImagesPullingTime="17.936499761s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:38.32675334 +0000 UTC m=+22.817805148" watchObservedRunningTime="2026-09-10 17:41:38.353579472 +0000 UTC m=+22.844631281"
Sep 10 17:41:38.379354 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.379304    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kdng8" podStartSLOduration=4.435722456 podStartE2EDuration="22.379290033s" totalImagesPullingTime="17.943567577s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:38.378979157 +0000 UTC m=+22.870030966" watchObservedRunningTime="2026-09-10 17:41:38.379290033 +0000 UTC m=+22.870341841"
Sep 10 17:41:38.379549 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.379532    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5trnt" podStartSLOduration=4.414776778 podStartE2EDuration="22.379528454s" totalImagesPullingTime="17.964751676s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:38.355061746 +0000 UTC m=+22.846113570" watchObservedRunningTime="2026-09-10 17:41:38.379528454 +0000 UTC m=+22.870580315"
Sep 10 17:41:38.541338 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:38.541313    2549 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 17:41:39.095330 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:39.095219    2549 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T17:41:38.541330878Z","UUID":"f2b49123-32f1-4a6b-b8ea-cb32941ade75","Handler":null,"Name":"","Endpoint":""}
Sep 10 17:41:39.096929 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:39.096906    2549 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 17:41:39.097054 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:39.096946    2549 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 17:41:39.257121 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:39.257085    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw" event={"ID":"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63","Type":"ContainerStarted","Data":"7c929f0243ca1544f22e84dbfa42d2851328c9e70c90d477fd69a5e5b2b3cac6"}
Sep 10 17:41:39.259408 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:39.259379    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-67z82" event={"ID":"8c013382-7f00-4396-802f-b54d938cff60","Type":"ContainerStarted","Data":"b7d8df027c9aced8d448815a1c1abe2c1ec26be71d30b9f77320ee8415e0903c"}
Sep 10 17:41:39.276948 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:39.276895    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-67z82" podStartSLOduration=5.337621701 podStartE2EDuration="23.276875564s" totalImagesPullingTime="17.939253863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:39.27610938 +0000 UTC m=+23.767161190" watchObservedRunningTime="2026-09-10 17:41:39.276875564 +0000 UTC m=+23.767927374"
Sep 10 17:41:40.023444 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:40.023232    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:40.023633 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:40.023247    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:40.023633 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:40.023554    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:40.023749 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:40.023644    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:40.023749 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:40.023308    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:40.023816 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:40.023752    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:40.265678 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:40.265640    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl" event={"ID":"50a3c219-25ae-42e8-9ed7-ae827178226a","Type":"ContainerStarted","Data":"36116dbf04e8624e3984dbb98a21b1246aa33b7100af5a3361bbc7d71d450cfb"}
Sep 10 17:41:40.267913 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:40.267874    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw" event={"ID":"9d16b6ee-5c09-4425-b3c2-fa4ee5366d63","Type":"ContainerStarted","Data":"6d475788017c2f2574f7cefc790e728b2932b8a23332be0548a525ec26668b45"}
Sep 10 17:41:40.286251 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:40.286162    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rhbhw" podStartSLOduration=4.159213363 podStartE2EDuration="24.286146956s" totalImagesPullingTime="20.126933593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:40.286134094 +0000 UTC m=+24.777185902" watchObservedRunningTime="2026-09-10 17:41:40.286146956 +0000 UTC m=+24.777198758"
Sep 10 17:41:41.192392 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:41.192356    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zbhnk"
Sep 10 17:41:41.192950 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:41.192926    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zbhnk"
Sep 10 17:41:42.023264 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:42.023227    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:42.023710 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:42.023273    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:42.023710 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:42.023227    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:42.023710 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:42.023359    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:42.023710 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:42.023463    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:42.023710 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:42.023547    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:43.279287 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:43.279088    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl" event={"ID":"50a3c219-25ae-42e8-9ed7-ae827178226a","Type":"ContainerStarted","Data":"deff7453a7726e6ddffe57ea19a5e7e386bbe330bd0fbfd30a42e6be6d483196"}
Sep 10 17:41:43.279712 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:43.279349    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:43.279712 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:43.279406    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:43.281016 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:43.280991    2549 generic.go:356] "Generic (PLEG): container finished" podID="213a7f60-9ca9-48fc-824e-6c5c14451aad" containerID="d4dcf85b8a414ea17e3fed88c0237a37fb155e158615a5ba7f8a51abb29a2b7f" exitCode=0
Sep 10 17:41:43.281139 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:43.281025    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lfxv" event={"ID":"213a7f60-9ca9-48fc-824e-6c5c14451aad","Type":"ContainerDied","Data":"d4dcf85b8a414ea17e3fed88c0237a37fb155e158615a5ba7f8a51abb29a2b7f"}
Sep 10 17:41:43.294682 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:43.294660    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:43.310929 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:43.310888    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl" podStartSLOduration=9.311887297 podStartE2EDuration="27.310870676s" totalImagesPullingTime="17.998983379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:43.310360703 +0000 UTC m=+27.801412511" watchObservedRunningTime="2026-09-10 17:41:43.310870676 +0000 UTC m=+27.801922485"
Sep 10 17:41:44.023078 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:44.023030    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:44.023214 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:44.023051    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:44.023214 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:44.023152    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:44.023214 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:44.023167    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:44.023390 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:44.023289    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:44.023390 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:44.023350    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:44.285842 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:44.285814    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lfxv" event={"ID":"213a7f60-9ca9-48fc-824e-6c5c14451aad","Type":"ContainerStarted","Data":"cf6e6a6bf3316cc59a298f973df3f715d4d21da974d5f839b0cd6b2cdddb3498"}
Sep 10 17:41:44.286382 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:44.286349    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:44.300036 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:44.300015    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:41:44.912129 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:44.912095    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mj85b"]
Sep 10 17:41:44.912285 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:44.912211    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:44.912342 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:44.912294    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:44.915384 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:44.915354    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5q47l"]
Sep 10 17:41:44.915509 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:44.915460    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:44.915582 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:44.915561    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:44.915953 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:44.915927    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tnrvx"]
Sep 10 17:41:44.916064 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:44.916012    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:44.916127 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:44.916103    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:45.290569 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:45.290536    2549 generic.go:356] "Generic (PLEG): container finished" podID="213a7f60-9ca9-48fc-824e-6c5c14451aad" containerID="cf6e6a6bf3316cc59a298f973df3f715d4d21da974d5f839b0cd6b2cdddb3498" exitCode=0
Sep 10 17:41:45.291164 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:45.290632    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lfxv" event={"ID":"213a7f60-9ca9-48fc-824e-6c5c14451aad","Type":"ContainerDied","Data":"cf6e6a6bf3316cc59a298f973df3f715d4d21da974d5f839b0cd6b2cdddb3498"}
Sep 10 17:41:46.295808 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:46.295610    2549 generic.go:356] "Generic (PLEG): container finished" podID="213a7f60-9ca9-48fc-824e-6c5c14451aad" containerID="10d465d88781c1f2bb5ad295ed3e34a70ec81ff337c5e6dddb2d587e3ac7c5f2" exitCode=0
Sep 10 17:41:46.296219 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:46.295688    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lfxv" event={"ID":"213a7f60-9ca9-48fc-824e-6c5c14451aad","Type":"ContainerDied","Data":"10d465d88781c1f2bb5ad295ed3e34a70ec81ff337c5e6dddb2d587e3ac7c5f2"}
Sep 10 17:41:47.022998 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:47.022962    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:47.023192 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:47.023084    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:47.023192 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:47.023095    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:47.023192 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:47.023133    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:47.023317 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:47.023240    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:47.023368 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:47.023335    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:47.474807 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:47.474769    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zbhnk"
Sep 10 17:41:47.475225 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:47.474911    2549 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 17:41:47.475461 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:47.475437    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zbhnk"
Sep 10 17:41:49.023403 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.023362    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:49.024003 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.023413    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:49.024003 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.023417    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:49.024003 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:49.023495    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5q47l" podUID="d61add86-4475-4f8f-b6d4-b3b298df34f4"
Sep 10 17:41:49.024003 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:49.023596    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mj85b" podUID="8cae49c8-331a-409b-8f6a-7d096c469ca5"
Sep 10 17:41:49.024003 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:49.023717    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tnrvx" podUID="78bdfa50-029c-4951-9ca4-4bfa31adf9a8"
Sep 10 17:41:49.649640 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.649602    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:49.649805 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:49.649771    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:41:49.649851 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:49.649836    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret podName:8cae49c8-331a-409b-8f6a-7d096c469ca5 nodeName:}" failed. No retries permitted until 2026-09-10 17:42:21.649819172 +0000 UTC m=+66.140870981 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret") pod "global-pull-secret-syncer-mj85b" (UID: "8cae49c8-331a-409b-8f6a-7d096c469ca5") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:41:49.750229 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.750188    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs\") pod \"network-metrics-daemon-tnrvx\" (UID: \"78bdfa50-029c-4951-9ca4-4bfa31adf9a8\") " pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:49.750390 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:49.750354    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:41:49.750455 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:49.750432    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs podName:78bdfa50-029c-4951-9ca4-4bfa31adf9a8 nodeName:}" failed. No retries permitted until 2026-09-10 17:42:21.750414286 +0000 UTC m=+66.241466073 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs") pod "network-metrics-daemon-tnrvx" (UID: "78bdfa50-029c-4951-9ca4-4bfa31adf9a8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:41:49.825982 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.825951    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-245.ec2.internal" event="NodeReady"
Sep 10 17:41:49.826167 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.826114    2549 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 17:41:49.851060 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.851021    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjtkq\" (UniqueName: \"kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq\") pod \"network-check-target-5q47l\" (UID: \"d61add86-4475-4f8f-b6d4-b3b298df34f4\") " pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:49.851257 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:49.851211    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:41:49.851257 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:49.851238    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:41:49.851257 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:49.851250    2549 projected.go:196] Error preparing data for projected volume kube-api-access-pjtkq for pod openshift-network-diagnostics/network-check-target-5q47l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:41:49.851404 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:49.851315    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq podName:d61add86-4475-4f8f-b6d4-b3b298df34f4 nodeName:}" failed. No retries permitted until 2026-09-10 17:42:21.851295784 +0000 UTC m=+66.342347581 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pjtkq" (UniqueName: "kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq") pod "network-check-target-5q47l" (UID: "d61add86-4475-4f8f-b6d4-b3b298df34f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:41:49.886673 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.886646    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kgrgd"]
Sep 10 17:41:49.930515 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.930429    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9gqnm"]
Sep 10 17:41:49.930668 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.930577    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:49.934274 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.934236    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 17:41:49.934420 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.934249    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 17:41:49.934420 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.934249    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d7j7h\""
Sep 10 17:41:49.951564 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.951528    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9gqnm"]
Sep 10 17:41:49.951564 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.951570    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kgrgd"]
Sep 10 17:41:49.951810 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.951660    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9gqnm"
Sep 10 17:41:49.957835 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.957779    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-p79lb\""
Sep 10 17:41:49.958147 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.958126    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 17:41:49.958274 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.958254    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 17:41:49.958274 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:49.958265    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 17:41:50.052657 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.052600    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36e2db1a-137d-45af-8cdd-3d2372c27efb-cert\") pod \"ingress-canary-9gqnm\" (UID: \"36e2db1a-137d-45af-8cdd-3d2372c27efb\") " pod="openshift-ingress-canary/ingress-canary-9gqnm"
Sep 10 17:41:50.052657 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.052658    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gqdk\" (UniqueName: \"kubernetes.io/projected/36e2db1a-137d-45af-8cdd-3d2372c27efb-kube-api-access-4gqdk\") pod \"ingress-canary-9gqnm\" (UID: \"36e2db1a-137d-45af-8cdd-3d2372c27efb\") " pod="openshift-ingress-canary/ingress-canary-9gqnm"
Sep 10 17:41:50.053131 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.052782    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/65e67805-ef05-4076-9cdc-71ece6139849-config-volume\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:50.053131 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.052810    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:50.053131 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.052832    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65e67805-ef05-4076-9cdc-71ece6139849-tmp-dir\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:50.053131 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.052892    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlj6w\" (UniqueName: \"kubernetes.io/projected/65e67805-ef05-4076-9cdc-71ece6139849-kube-api-access-jlj6w\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:50.154138 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.154082    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36e2db1a-137d-45af-8cdd-3d2372c27efb-cert\") pod \"ingress-canary-9gqnm\" (UID: \"36e2db1a-137d-45af-8cdd-3d2372c27efb\") " pod="openshift-ingress-canary/ingress-canary-9gqnm"
Sep 10 17:41:50.154138 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.154142    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gqdk\" (UniqueName: \"kubernetes.io/projected/36e2db1a-137d-45af-8cdd-3d2372c27efb-kube-api-access-4gqdk\") pod \"ingress-canary-9gqnm\" (UID: \"36e2db1a-137d-45af-8cdd-3d2372c27efb\") " pod="openshift-ingress-canary/ingress-canary-9gqnm"
Sep 10 17:41:50.154380 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.154186    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/65e67805-ef05-4076-9cdc-71ece6139849-config-volume\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:50.154380 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.154204    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:50.154380 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.154217    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65e67805-ef05-4076-9cdc-71ece6139849-tmp-dir\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:50.154380 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.154240    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlj6w\" (UniqueName: \"kubernetes.io/projected/65e67805-ef05-4076-9cdc-71ece6139849-kube-api-access-jlj6w\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:50.154380 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:50.154311    2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 17:41:50.154380 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:50.154379    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls podName:65e67805-ef05-4076-9cdc-71ece6139849 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:50.654358859 +0000 UTC m=+35.145410665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls") pod "dns-default-kgrgd" (UID: "65e67805-ef05-4076-9cdc-71ece6139849") : secret "dns-default-metrics-tls" not found
Sep 10 17:41:50.154775 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.154640    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65e67805-ef05-4076-9cdc-71ece6139849-tmp-dir\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:50.154835 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.154817    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/65e67805-ef05-4076-9cdc-71ece6139849-config-volume\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:50.159058 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.159030    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36e2db1a-137d-45af-8cdd-3d2372c27efb-cert\") pod \"ingress-canary-9gqnm\" (UID: \"36e2db1a-137d-45af-8cdd-3d2372c27efb\") " pod="openshift-ingress-canary/ingress-canary-9gqnm"
Sep 10 17:41:50.169776 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.169743    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gqdk\" (UniqueName: \"kubernetes.io/projected/36e2db1a-137d-45af-8cdd-3d2372c27efb-kube-api-access-4gqdk\") pod \"ingress-canary-9gqnm\" (UID: \"36e2db1a-137d-45af-8cdd-3d2372c27efb\") " pod="openshift-ingress-canary/ingress-canary-9gqnm"
Sep 10 17:41:50.171165 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.171141    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlj6w\" (UniqueName: \"kubernetes.io/projected/65e67805-ef05-4076-9cdc-71ece6139849-kube-api-access-jlj6w\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:50.262507 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.262473    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9gqnm"
Sep 10 17:41:50.658708 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.658600    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:50.658854 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:50.658783    2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 17:41:50.658854 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:50.658848    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls podName:65e67805-ef05-4076-9cdc-71ece6139849 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:51.658831756 +0000 UTC m=+36.149883561 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls") pod "dns-default-kgrgd" (UID: "65e67805-ef05-4076-9cdc-71ece6139849") : secret "dns-default-metrics-tls" not found
Sep 10 17:41:50.709793 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:50.709760    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9gqnm"]
Sep 10 17:41:50.713293 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:50.713264    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36e2db1a_137d_45af_8cdd_3d2372c27efb.slice/crio-18c20167ec46bf87e606ba923a7b7b7c8b4bbce99123603b449b596defa2ed4a WatchSource:0}: Error finding container 18c20167ec46bf87e606ba923a7b7b7c8b4bbce99123603b449b596defa2ed4a: Status 404 returned error can't find the container with id 18c20167ec46bf87e606ba923a7b7b7c8b4bbce99123603b449b596defa2ed4a
Sep 10 17:41:51.023008 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:51.022973    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:41:51.023189 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:51.023073    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:41:51.023189 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:51.023107    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:41:51.027518 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:51.027497    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 17:41:51.027640 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:51.027499    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 17:41:51.028555 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:51.028533    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 17:41:51.028671 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:51.028537    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fbgqn\""
Sep 10 17:41:51.028671 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:51.028543    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 17:41:51.028866 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:51.028852    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hkln9\""
Sep 10 17:41:51.308861 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:51.308829    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9gqnm" event={"ID":"36e2db1a-137d-45af-8cdd-3d2372c27efb","Type":"ContainerStarted","Data":"18c20167ec46bf87e606ba923a7b7b7c8b4bbce99123603b449b596defa2ed4a"}
Sep 10 17:41:51.311424 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:51.311400    2549 generic.go:356] "Generic (PLEG): container finished" podID="213a7f60-9ca9-48fc-824e-6c5c14451aad" containerID="9bd0757f53b9a5e768b51e2d335955c5ff128140816dd84a76b830ac0ef6d958" exitCode=0
Sep 10 17:41:51.311526 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:51.311438    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lfxv" event={"ID":"213a7f60-9ca9-48fc-824e-6c5c14451aad","Type":"ContainerDied","Data":"9bd0757f53b9a5e768b51e2d335955c5ff128140816dd84a76b830ac0ef6d958"}
Sep 10 17:41:51.668329 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:51.668231    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:51.668486 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:51.668351    2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 17:41:51.668486 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:51.668428    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls podName:65e67805-ef05-4076-9cdc-71ece6139849 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:53.668405724 +0000 UTC m=+38.159457541 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls") pod "dns-default-kgrgd" (UID: "65e67805-ef05-4076-9cdc-71ece6139849") : secret "dns-default-metrics-tls" not found
Sep 10 17:41:52.317681 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:52.317645    2549 generic.go:356] "Generic (PLEG): container finished" podID="213a7f60-9ca9-48fc-824e-6c5c14451aad" containerID="8064d20e35d79d4baf310a7e9e4cf181466cc1d5ec90bbd5d36fbabf7d4674f9" exitCode=0
Sep 10 17:41:52.318217 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:52.317678    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lfxv" event={"ID":"213a7f60-9ca9-48fc-824e-6c5c14451aad","Type":"ContainerDied","Data":"8064d20e35d79d4baf310a7e9e4cf181466cc1d5ec90bbd5d36fbabf7d4674f9"}
Sep 10 17:41:53.323065 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:53.323029    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lfxv" event={"ID":"213a7f60-9ca9-48fc-824e-6c5c14451aad","Type":"ContainerStarted","Data":"ba29ead035c6dc8f738b31376d8468cf3565eebd065bd1fbe78dd2cd95071770"}
Sep 10 17:41:53.348005 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:53.347950    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9lfxv" podStartSLOduration=8.792825639 podStartE2EDuration="37.347933062s" totalImagesPullingTime="28.555107423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:53.346140047 +0000 UTC m=+37.837191856" watchObservedRunningTime="2026-09-10 17:41:53.347933062 +0000 UTC m=+37.838984872"
Sep 10 17:41:53.683863 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:53.683649    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:53.684049 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:53.683795    2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 17:41:53.684049 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:53.683986    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls podName:65e67805-ef05-4076-9cdc-71ece6139849 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:57.683969972 +0000 UTC m=+42.175021763 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls") pod "dns-default-kgrgd" (UID: "65e67805-ef05-4076-9cdc-71ece6139849") : secret "dns-default-metrics-tls" not found
Sep 10 17:41:54.326602 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.326560    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9gqnm" event={"ID":"36e2db1a-137d-45af-8cdd-3d2372c27efb","Type":"ContainerStarted","Data":"5572956db798c9b08192b6f6b3cfe56a780e39a392023a6f208e3ffcdea4f3dd"}
Sep 10 17:41:54.348861 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.348814    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9gqnm" podStartSLOduration=2.582153343 podStartE2EDuration="5.348798692s" totalImagesPullingTime="2.766645349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:54.348103943 +0000 UTC m=+38.839155752" watchObservedRunningTime="2026-09-10 17:41:54.348798692 +0000 UTC m=+38.839850500"
Sep 10 17:41:54.785855 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.785819    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v92d8"]
Sep 10 17:41:54.822862 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.822832    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v92d8"]
Sep 10 17:41:54.823006 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.822942    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:54.826263 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.826241    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 17:41:54.826263 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.826257    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 17:41:54.826436 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.826422    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fk6pf\""
Sep 10 17:41:54.826484 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.826476    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 17:41:54.827376 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.827364    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 17:41:54.948562 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.948526    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7hrx5"]
Sep 10 17:41:54.970029 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.970002    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7hrx5"]
Sep 10 17:41:54.970184 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.970125    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7hrx5"
Sep 10 17:41:54.973115 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.973097    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 17:41:54.973115 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.973111    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 17:41:54.973437 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.973421    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-q7fkg\""
Sep 10 17:41:54.994336 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.994308    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:54.994336 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.994337    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zltpk\" (UniqueName: \"kubernetes.io/projected/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-kube-api-access-zltpk\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:54.994518 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.994358    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-data-volume\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:54.994518 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.994376    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:54.994518 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:54.994488    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-crio-socket\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:55.094974 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.094890    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-crio-socket\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:55.094974 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.094943    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:55.094974 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.094962    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j99wb\" (UniqueName: \"kubernetes.io/projected/c1c55e61-318c-4184-a18c-f9505f66f332-kube-api-access-j99wb\") pod \"migrator-fdbb8fdff-7hrx5\" (UID: \"c1c55e61-318c-4184-a18c-f9505f66f332\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7hrx5"
Sep 10 17:41:55.095221 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.095020    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zltpk\" (UniqueName: \"kubernetes.io/projected/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-kube-api-access-zltpk\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:55.095221 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:55.095037    2549 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 17:41:55.095221 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.095056    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-data-volume\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:55.095221 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.095076    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:55.095221 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:55.095097    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-kube-rbac-proxy-cm podName:defc63f5-ba34-4dd3-bb22-fd991d6be1ac nodeName:}" failed. No retries permitted until 2026-09-10 17:41:55.59508129 +0000 UTC m=+40.086133101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v92d8" (UID: "defc63f5-ba34-4dd3-bb22-fd991d6be1ac") : configmap "kube-rbac-proxy" not found
Sep 10 17:41:55.095221 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:55.095156    2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 17:41:55.095221 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.095175    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-crio-socket\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:55.095221 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:55.095199    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls podName:defc63f5-ba34-4dd3-bb22-fd991d6be1ac nodeName:}" failed. No retries permitted until 2026-09-10 17:41:55.595186808 +0000 UTC m=+40.086238595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v92d8" (UID: "defc63f5-ba34-4dd3-bb22-fd991d6be1ac") : secret "insights-runtime-extractor-tls" not found
Sep 10 17:41:55.095556 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.095344    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-data-volume\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:55.122101 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.122060    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zltpk\" (UniqueName: \"kubernetes.io/projected/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-kube-api-access-zltpk\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:55.196139 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.196099    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j99wb\" (UniqueName: \"kubernetes.io/projected/c1c55e61-318c-4184-a18c-f9505f66f332-kube-api-access-j99wb\") pod \"migrator-fdbb8fdff-7hrx5\" (UID: \"c1c55e61-318c-4184-a18c-f9505f66f332\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7hrx5"
Sep 10 17:41:55.215573 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.215546    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j99wb\" (UniqueName: \"kubernetes.io/projected/c1c55e61-318c-4184-a18c-f9505f66f332-kube-api-access-j99wb\") pod \"migrator-fdbb8fdff-7hrx5\" (UID: \"c1c55e61-318c-4184-a18c-f9505f66f332\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7hrx5"
Sep 10 17:41:55.278938 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.278904    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7hrx5"
Sep 10 17:41:55.559321 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.559279    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7hrx5"]
Sep 10 17:41:55.561897 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:41:55.561869    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1c55e61_318c_4184_a18c_f9505f66f332.slice/crio-35899d42ebdf3a1ec365460e50a014954b1b0bbba19eeff1d92557443195d5e6 WatchSource:0}: Error finding container 35899d42ebdf3a1ec365460e50a014954b1b0bbba19eeff1d92557443195d5e6: Status 404 returned error can't find the container with id 35899d42ebdf3a1ec365460e50a014954b1b0bbba19eeff1d92557443195d5e6
Sep 10 17:41:55.598890 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.598866    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:55.598988 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.598904    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:55.599040 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:55.599025    2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 17:41:55.599099 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:55.599088    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls podName:defc63f5-ba34-4dd3-bb22-fd991d6be1ac nodeName:}" failed. No retries permitted until 2026-09-10 17:41:56.599070024 +0000 UTC m=+41.090121810 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v92d8" (UID: "defc63f5-ba34-4dd3-bb22-fd991d6be1ac") : secret "insights-runtime-extractor-tls" not found
Sep 10 17:41:55.599479 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:55.599463    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:56.334142 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:56.334054    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7hrx5" event={"ID":"c1c55e61-318c-4184-a18c-f9505f66f332","Type":"ContainerStarted","Data":"35899d42ebdf3a1ec365460e50a014954b1b0bbba19eeff1d92557443195d5e6"}
Sep 10 17:41:56.606458 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:56.606358    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:56.606922 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:56.606534    2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 17:41:56.606922 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:56.606625    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls podName:defc63f5-ba34-4dd3-bb22-fd991d6be1ac nodeName:}" failed. No retries permitted until 2026-09-10 17:41:58.606597943 +0000 UTC m=+43.097649742 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v92d8" (UID: "defc63f5-ba34-4dd3-bb22-fd991d6be1ac") : secret "insights-runtime-extractor-tls" not found
Sep 10 17:41:56.935687 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:56.935611    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kdng8_df747e4e-bb42-420a-90e5-318f0b4f31e8/dns-node-resolver/0.log"
Sep 10 17:41:57.337526 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:57.337498    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7hrx5" event={"ID":"c1c55e61-318c-4184-a18c-f9505f66f332","Type":"ContainerStarted","Data":"14e5e8d880c5a3cf5b88400765dc7e9fffcf0e8ac4660dec0eb37c3f08a3bc36"}
Sep 10 17:41:57.714863 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:57.714819    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:41:57.715291 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:57.714978    2549 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 17:41:57.715291 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:57.715054    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls podName:65e67805-ef05-4076-9cdc-71ece6139849 nodeName:}" failed. No retries permitted until 2026-09-10 17:42:05.715038859 +0000 UTC m=+50.206090668 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls") pod "dns-default-kgrgd" (UID: "65e67805-ef05-4076-9cdc-71ece6139849") : secret "dns-default-metrics-tls" not found
Sep 10 17:41:57.735612 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:57.735594    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jpdgj_1c14a9e5-fec4-4b25-b3a2-99d9bb6be063/node-ca/0.log"
Sep 10 17:41:58.341870 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:58.341834    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7hrx5" event={"ID":"c1c55e61-318c-4184-a18c-f9505f66f332","Type":"ContainerStarted","Data":"941e13820685ff09ead38bbe7e8d84125f98f601d89fd3fbfd081d40ad834193"}
Sep 10 17:41:58.535387 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:58.535358    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9gqnm_36e2db1a-137d-45af-8cdd-3d2372c27efb/serve-healthcheck-canary/0.log"
Sep 10 17:41:58.620912 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:41:58.620840    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:41:58.621017 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:58.620959    2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 17:41:58.621017 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:41:58.621008    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls podName:defc63f5-ba34-4dd3-bb22-fd991d6be1ac nodeName:}" failed. No retries permitted until 2026-09-10 17:42:02.620995572 +0000 UTC m=+47.112047358 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v92d8" (UID: "defc63f5-ba34-4dd3-bb22-fd991d6be1ac") : secret "insights-runtime-extractor-tls" not found
Sep 10 17:42:02.651004 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:02.650966    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:42:02.651389 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:42:02.651074    2549 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 17:42:02.651389 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:42:02.651136    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls podName:defc63f5-ba34-4dd3-bb22-fd991d6be1ac nodeName:}" failed. No retries permitted until 2026-09-10 17:42:10.651122868 +0000 UTC m=+55.142174654 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v92d8" (UID: "defc63f5-ba34-4dd3-bb22-fd991d6be1ac") : secret "insights-runtime-extractor-tls" not found
Sep 10 17:42:05.774293 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:05.774254    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:42:05.776680 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:05.776652    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/65e67805-ef05-4076-9cdc-71ece6139849-metrics-tls\") pod \"dns-default-kgrgd\" (UID: \"65e67805-ef05-4076-9cdc-71ece6139849\") " pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:42:05.841235 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:05.841193    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:42:06.134113 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:06.134000    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7hrx5" podStartSLOduration=10.558097631999999 podStartE2EDuration="12.133982307s" totalImagesPullingTime="1.575884675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:58.357395572 +0000 UTC m=+42.848447382" watchObservedRunningTime="2026-09-10 17:42:06.133982307 +0000 UTC m=+50.625034111"
Sep 10 17:42:06.134436 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:06.134418    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kgrgd"]
Sep 10 17:42:06.137331 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:42:06.137308    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e67805_ef05_4076_9cdc_71ece6139849.slice/crio-05d815f26911e812c82b9a20ad305a6e31e266d6b1c10e4cf22db8f03bc77c3b WatchSource:0}: Error finding container 05d815f26911e812c82b9a20ad305a6e31e266d6b1c10e4cf22db8f03bc77c3b: Status 404 returned error can't find the container with id 05d815f26911e812c82b9a20ad305a6e31e266d6b1c10e4cf22db8f03bc77c3b
Sep 10 17:42:06.362774 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:06.362738    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kgrgd" event={"ID":"65e67805-ef05-4076-9cdc-71ece6139849","Type":"ContainerStarted","Data":"05d815f26911e812c82b9a20ad305a6e31e266d6b1c10e4cf22db8f03bc77c3b"}
Sep 10 17:42:08.370804 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:08.370771    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kgrgd" event={"ID":"65e67805-ef05-4076-9cdc-71ece6139849","Type":"ContainerStarted","Data":"bd26f4ff732fe60cb3ad4cdede348ed7ce0723079d434a7529b5f89dee0bfb05"}
Sep 10 17:42:09.375402 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:09.375362    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kgrgd" event={"ID":"65e67805-ef05-4076-9cdc-71ece6139849","Type":"ContainerStarted","Data":"63e8042052adf603bc6f5b1b64e238e823c1ba67a4e0be5389d8edcf37cc57b4"}
Sep 10 17:42:09.375864 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:09.375448    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:42:09.397409 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:09.397362    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kgrgd" podStartSLOduration=18.266906749 podStartE2EDuration="20.397348403s" totalImagesPullingTime="2.130441654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:09.396795565 +0000 UTC m=+53.887847378" watchObservedRunningTime="2026-09-10 17:42:09.397348403 +0000 UTC m=+53.888400212"
Sep 10 17:42:10.710855 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:10.710816    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:42:10.713042 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:10.713014    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/defc63f5-ba34-4dd3-bb22-fd991d6be1ac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v92d8\" (UID: \"defc63f5-ba34-4dd3-bb22-fd991d6be1ac\") " pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:42:10.731978 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:10.731955    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v92d8"
Sep 10 17:42:11.007730 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:11.007632    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v92d8"]
Sep 10 17:42:11.011373 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:42:11.011338    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddefc63f5_ba34_4dd3_bb22_fd991d6be1ac.slice/crio-10d11f9a772acbfb2f74f6372ef334cce0cd8267c6fa6e878a9acc8f264b8869 WatchSource:0}: Error finding container 10d11f9a772acbfb2f74f6372ef334cce0cd8267c6fa6e878a9acc8f264b8869: Status 404 returned error can't find the container with id 10d11f9a772acbfb2f74f6372ef334cce0cd8267c6fa6e878a9acc8f264b8869
Sep 10 17:42:11.383458 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:11.383378    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v92d8" event={"ID":"defc63f5-ba34-4dd3-bb22-fd991d6be1ac","Type":"ContainerStarted","Data":"c7c5672852472a9b9a108320336f5cf298b0b0dd20fd2074f75a12b71d7a283e"}
Sep 10 17:42:11.383458 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:11.383433    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v92d8" event={"ID":"defc63f5-ba34-4dd3-bb22-fd991d6be1ac","Type":"ContainerStarted","Data":"10d11f9a772acbfb2f74f6372ef334cce0cd8267c6fa6e878a9acc8f264b8869"}
Sep 10 17:42:12.389225 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:12.389185    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v92d8" event={"ID":"defc63f5-ba34-4dd3-bb22-fd991d6be1ac","Type":"ContainerStarted","Data":"bbd1babc7e75adcdcfd3c27705b2a57d7b990aaf376d2ea8bcbcc6cd736c6d17"}
Sep 10 17:42:13.394647 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:13.394606    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v92d8" event={"ID":"defc63f5-ba34-4dd3-bb22-fd991d6be1ac","Type":"ContainerStarted","Data":"8bd5eddd451b43591eb41a52a5912dc1ddd930ac92b9ac07b81c3d34a3be1d7e"}
Sep 10 17:42:13.415882 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:13.415826    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v92d8" podStartSLOduration=17.660431177 podStartE2EDuration="19.415810542s" totalImagesPullingTime="1.755379365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:13.414438141 +0000 UTC m=+57.905489946" watchObservedRunningTime="2026-09-10 17:42:13.415810542 +0000 UTC m=+57.906862349"
Sep 10 17:42:16.306948 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:16.306916    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-krlkl"
Sep 10 17:42:17.514389 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.514350    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"]
Sep 10 17:42:17.517199 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.517181    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-558b47745b-m8cc6"]
Sep 10 17:42:17.517325 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.517305    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"
Sep 10 17:42:17.519134 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.519111    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-558b47745b-m8cc6"
Sep 10 17:42:17.520159 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.520134    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 17:42:17.520480 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.520463    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 17:42:17.521598 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.521576    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 17:42:17.521722 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.521598    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 10 17:42:17.521956 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.521937    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-bgx9p\""
Sep 10 17:42:17.522033 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.521942    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 10 17:42:17.533097 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.533077    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"]
Sep 10 17:42:17.534342 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.534320    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-558b47745b-m8cc6"]
Sep 10 17:42:17.561575 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.561540    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7aee70cf-530b-45b8-bd3f-819391da6d1e-klusterlet-config\") pod \"klusterlet-addon-workmgr-dd8798dd4-6mdcf\" (UID: \"7aee70cf-530b-45b8-bd3f-819391da6d1e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"
Sep 10 17:42:17.561779 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.561604    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0c62b3e5-82e1-4608-bfaa-815ca05f9bee-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-558b47745b-m8cc6\" (UID: \"0c62b3e5-82e1-4608-bfaa-815ca05f9bee\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-558b47745b-m8cc6"
Sep 10 17:42:17.561779 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.561643    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgr2s\" (UniqueName: \"kubernetes.io/projected/0c62b3e5-82e1-4608-bfaa-815ca05f9bee-kube-api-access-xgr2s\") pod \"managed-serviceaccount-addon-agent-558b47745b-m8cc6\" (UID: \"0c62b3e5-82e1-4608-bfaa-815ca05f9bee\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-558b47745b-m8cc6"
Sep 10 17:42:17.561779 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.561678    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tcc7\" (UniqueName: \"kubernetes.io/projected/7aee70cf-530b-45b8-bd3f-819391da6d1e-kube-api-access-5tcc7\") pod \"klusterlet-addon-workmgr-dd8798dd4-6mdcf\" (UID: \"7aee70cf-530b-45b8-bd3f-819391da6d1e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"
Sep 10 17:42:17.561779 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.561725    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7aee70cf-530b-45b8-bd3f-819391da6d1e-tmp\") pod \"klusterlet-addon-workmgr-dd8798dd4-6mdcf\" (UID: \"7aee70cf-530b-45b8-bd3f-819391da6d1e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"
Sep 10 17:42:17.616288 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.616251    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-58d4769c69-rjtx9"]
Sep 10 17:42:17.618007 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.617988    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-4nnsl"]
Sep 10 17:42:17.618163 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.618144    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.620102 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.620083    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-4nnsl"
Sep 10 17:42:17.621802 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.621785    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 17:42:17.623811 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.623788    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 17:42:17.624090 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.624071    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 17:42:17.624176 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.624074    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-stkqn\""
Sep 10 17:42:17.624611 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.624577    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 17:42:17.624771 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.624596    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 17:42:17.624848 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.624788    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-bbblv\""
Sep 10 17:42:17.632620 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.632598    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 17:42:17.646079 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.646054    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-4nnsl"]
Sep 10 17:42:17.648920 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.648899    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58d4769c69-rjtx9"]
Sep 10 17:42:17.662402 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.662377    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7aee70cf-530b-45b8-bd3f-819391da6d1e-klusterlet-config\") pod \"klusterlet-addon-workmgr-dd8798dd4-6mdcf\" (UID: \"7aee70cf-530b-45b8-bd3f-819391da6d1e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"
Sep 10 17:42:17.662535 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.662412    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fff37c0e-0b99-46d0-afc5-a6e2132641f6-image-registry-private-configuration\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.662535 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.662436    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs2pr\" (UniqueName: \"kubernetes.io/projected/b459cafb-dae0-4cfd-9859-ed15e033dc5c-kube-api-access-cs2pr\") pod \"downloads-85d98c998f-4nnsl\" (UID: \"b459cafb-dae0-4cfd-9859-ed15e033dc5c\") " pod="openshift-console/downloads-85d98c998f-4nnsl"
Sep 10 17:42:17.662535 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.662457    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fff37c0e-0b99-46d0-afc5-a6e2132641f6-bound-sa-token\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.662535 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.662484    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5m74\" (UniqueName: \"kubernetes.io/projected/fff37c0e-0b99-46d0-afc5-a6e2132641f6-kube-api-access-c5m74\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.662826 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.662636    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0c62b3e5-82e1-4608-bfaa-815ca05f9bee-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-558b47745b-m8cc6\" (UID: \"0c62b3e5-82e1-4608-bfaa-815ca05f9bee\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-558b47745b-m8cc6"
Sep 10 17:42:17.662826 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.662667    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fff37c0e-0b99-46d0-afc5-a6e2132641f6-ca-trust-extracted\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.662826 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.662712    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fff37c0e-0b99-46d0-afc5-a6e2132641f6-registry-certificates\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.662826 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.662740    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgr2s\" (UniqueName: \"kubernetes.io/projected/0c62b3e5-82e1-4608-bfaa-815ca05f9bee-kube-api-access-xgr2s\") pod \"managed-serviceaccount-addon-agent-558b47745b-m8cc6\" (UID: \"0c62b3e5-82e1-4608-bfaa-815ca05f9bee\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-558b47745b-m8cc6"
Sep 10 17:42:17.662826 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.662764    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fff37c0e-0b99-46d0-afc5-a6e2132641f6-registry-tls\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.662826 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.662785    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fff37c0e-0b99-46d0-afc5-a6e2132641f6-trusted-ca\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.662826 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.662815    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tcc7\" (UniqueName: \"kubernetes.io/projected/7aee70cf-530b-45b8-bd3f-819391da6d1e-kube-api-access-5tcc7\") pod \"klusterlet-addon-workmgr-dd8798dd4-6mdcf\" (UID: \"7aee70cf-530b-45b8-bd3f-819391da6d1e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"
Sep 10 17:42:17.663212 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.662854    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fff37c0e-0b99-46d0-afc5-a6e2132641f6-installation-pull-secrets\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.663212 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.662961    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7aee70cf-530b-45b8-bd3f-819391da6d1e-tmp\") pod \"klusterlet-addon-workmgr-dd8798dd4-6mdcf\" (UID: \"7aee70cf-530b-45b8-bd3f-819391da6d1e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"
Sep 10 17:42:17.663346 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.663299    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7aee70cf-530b-45b8-bd3f-819391da6d1e-tmp\") pod \"klusterlet-addon-workmgr-dd8798dd4-6mdcf\" (UID: \"7aee70cf-530b-45b8-bd3f-819391da6d1e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"
Sep 10 17:42:17.665182 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.665156    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7aee70cf-530b-45b8-bd3f-819391da6d1e-klusterlet-config\") pod \"klusterlet-addon-workmgr-dd8798dd4-6mdcf\" (UID: \"7aee70cf-530b-45b8-bd3f-819391da6d1e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"
Sep 10 17:42:17.665468 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.665451    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0c62b3e5-82e1-4608-bfaa-815ca05f9bee-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-558b47745b-m8cc6\" (UID: \"0c62b3e5-82e1-4608-bfaa-815ca05f9bee\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-558b47745b-m8cc6"
Sep 10 17:42:17.675509 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.675480    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tcc7\" (UniqueName: \"kubernetes.io/projected/7aee70cf-530b-45b8-bd3f-819391da6d1e-kube-api-access-5tcc7\") pod \"klusterlet-addon-workmgr-dd8798dd4-6mdcf\" (UID: \"7aee70cf-530b-45b8-bd3f-819391da6d1e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"
Sep 10 17:42:17.676165 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.676148    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgr2s\" (UniqueName: \"kubernetes.io/projected/0c62b3e5-82e1-4608-bfaa-815ca05f9bee-kube-api-access-xgr2s\") pod \"managed-serviceaccount-addon-agent-558b47745b-m8cc6\" (UID: \"0c62b3e5-82e1-4608-bfaa-815ca05f9bee\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-558b47745b-m8cc6"
Sep 10 17:42:17.764139 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.764107    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs2pr\" (UniqueName: \"kubernetes.io/projected/b459cafb-dae0-4cfd-9859-ed15e033dc5c-kube-api-access-cs2pr\") pod \"downloads-85d98c998f-4nnsl\" (UID: \"b459cafb-dae0-4cfd-9859-ed15e033dc5c\") " pod="openshift-console/downloads-85d98c998f-4nnsl"
Sep 10 17:42:17.764139 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.764140    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fff37c0e-0b99-46d0-afc5-a6e2132641f6-bound-sa-token\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.764421 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.764158    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5m74\" (UniqueName: \"kubernetes.io/projected/fff37c0e-0b99-46d0-afc5-a6e2132641f6-kube-api-access-c5m74\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.764421 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.764328    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fff37c0e-0b99-46d0-afc5-a6e2132641f6-ca-trust-extracted\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.764421 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.764359    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fff37c0e-0b99-46d0-afc5-a6e2132641f6-registry-certificates\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.764421 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.764384    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fff37c0e-0b99-46d0-afc5-a6e2132641f6-registry-tls\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.764421 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.764407    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fff37c0e-0b99-46d0-afc5-a6e2132641f6-trusted-ca\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.764658 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.764432    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fff37c0e-0b99-46d0-afc5-a6e2132641f6-installation-pull-secrets\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.764658 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.764495    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fff37c0e-0b99-46d0-afc5-a6e2132641f6-image-registry-private-configuration\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.764902 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.764874    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fff37c0e-0b99-46d0-afc5-a6e2132641f6-ca-trust-extracted\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.765306 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.765285    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fff37c0e-0b99-46d0-afc5-a6e2132641f6-registry-certificates\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.766066 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.766038    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fff37c0e-0b99-46d0-afc5-a6e2132641f6-trusted-ca\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.766995 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.766974    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fff37c0e-0b99-46d0-afc5-a6e2132641f6-installation-pull-secrets\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.767081 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.766996    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fff37c0e-0b99-46d0-afc5-a6e2132641f6-registry-tls\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.767405 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.767376    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fff37c0e-0b99-46d0-afc5-a6e2132641f6-image-registry-private-configuration\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.775153 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.775129    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5m74\" (UniqueName: \"kubernetes.io/projected/fff37c0e-0b99-46d0-afc5-a6e2132641f6-kube-api-access-c5m74\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.775232 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.775207    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs2pr\" (UniqueName: \"kubernetes.io/projected/b459cafb-dae0-4cfd-9859-ed15e033dc5c-kube-api-access-cs2pr\") pod \"downloads-85d98c998f-4nnsl\" (UID: \"b459cafb-dae0-4cfd-9859-ed15e033dc5c\") " pod="openshift-console/downloads-85d98c998f-4nnsl"
Sep 10 17:42:17.775377 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.775357    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fff37c0e-0b99-46d0-afc5-a6e2132641f6-bound-sa-token\") pod \"image-registry-58d4769c69-rjtx9\" (UID: \"fff37c0e-0b99-46d0-afc5-a6e2132641f6\") " pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.828264 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.828232    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"
Sep 10 17:42:17.843906 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.843882    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-558b47745b-m8cc6"
Sep 10 17:42:17.930248 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.930221    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:17.935784 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:17.935761    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-4nnsl"
Sep 10 17:42:18.098879 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:18.098837    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"]
Sep 10 17:42:18.100872 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:42:18.100846    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aee70cf_530b_45b8_bd3f_819391da6d1e.slice/crio-eda451aec4670c7024eb3b002872d008a86a5f32a8b353f3fb10000b2ebead93 WatchSource:0}: Error finding container eda451aec4670c7024eb3b002872d008a86a5f32a8b353f3fb10000b2ebead93: Status 404 returned error can't find the container with id eda451aec4670c7024eb3b002872d008a86a5f32a8b353f3fb10000b2ebead93
Sep 10 17:42:18.118404 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:18.118381    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-558b47745b-m8cc6"]
Sep 10 17:42:18.121082 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:42:18.121063    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c62b3e5_82e1_4608_bfaa_815ca05f9bee.slice/crio-5f7ce652a2f5f70f04fb74ffddfab6a1b91e260eb3fd6ab4b363c4d2293e524d WatchSource:0}: Error finding container 5f7ce652a2f5f70f04fb74ffddfab6a1b91e260eb3fd6ab4b363c4d2293e524d: Status 404 returned error can't find the container with id 5f7ce652a2f5f70f04fb74ffddfab6a1b91e260eb3fd6ab4b363c4d2293e524d
Sep 10 17:42:18.210831 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:18.210803    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58d4769c69-rjtx9"]
Sep 10 17:42:18.213599 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:42:18.213564    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfff37c0e_0b99_46d0_afc5_a6e2132641f6.slice/crio-45ebcb231b3790b653a468e75c58c349dcb168a8b58248ca9311bed119f39a86 WatchSource:0}: Error finding container 45ebcb231b3790b653a468e75c58c349dcb168a8b58248ca9311bed119f39a86: Status 404 returned error can't find the container with id 45ebcb231b3790b653a468e75c58c349dcb168a8b58248ca9311bed119f39a86
Sep 10 17:42:18.218325 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:18.218297    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-4nnsl"]
Sep 10 17:42:18.221406 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:42:18.221387    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb459cafb_dae0_4cfd_9859_ed15e033dc5c.slice/crio-09510c6be791e4cbf2f6a735f1ab791d4232b089224367810aeb4b80a2da32a5 WatchSource:0}: Error finding container 09510c6be791e4cbf2f6a735f1ab791d4232b089224367810aeb4b80a2da32a5: Status 404 returned error can't find the container with id 09510c6be791e4cbf2f6a735f1ab791d4232b089224367810aeb4b80a2da32a5
Sep 10 17:42:18.410636 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:18.410533    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58d4769c69-rjtx9" event={"ID":"fff37c0e-0b99-46d0-afc5-a6e2132641f6","Type":"ContainerStarted","Data":"d0668d2fc0e6fd4b3e877f899dffac35d97b38aaecd8f68dca852eb44caa640b"}
Sep 10 17:42:18.410636 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:18.410590    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58d4769c69-rjtx9" event={"ID":"fff37c0e-0b99-46d0-afc5-a6e2132641f6","Type":"ContainerStarted","Data":"45ebcb231b3790b653a468e75c58c349dcb168a8b58248ca9311bed119f39a86"}
Sep 10 17:42:18.410636 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:18.410624    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:18.411771 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:18.411745    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-558b47745b-m8cc6" event={"ID":"0c62b3e5-82e1-4608-bfaa-815ca05f9bee","Type":"ContainerStarted","Data":"5f7ce652a2f5f70f04fb74ffddfab6a1b91e260eb3fd6ab4b363c4d2293e524d"}
Sep 10 17:42:18.412923 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:18.412902    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf" event={"ID":"7aee70cf-530b-45b8-bd3f-819391da6d1e","Type":"ContainerStarted","Data":"eda451aec4670c7024eb3b002872d008a86a5f32a8b353f3fb10000b2ebead93"}
Sep 10 17:42:18.414038 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:18.414018    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-4nnsl" event={"ID":"b459cafb-dae0-4cfd-9859-ed15e033dc5c","Type":"ContainerStarted","Data":"09510c6be791e4cbf2f6a735f1ab791d4232b089224367810aeb4b80a2da32a5"}
Sep 10 17:42:18.433856 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:18.433809    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-58d4769c69-rjtx9" podStartSLOduration=1.433792717 podStartE2EDuration="1.433792717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:18.433569305 +0000 UTC m=+62.924621145" watchObservedRunningTime="2026-09-10 17:42:18.433792717 +0000 UTC m=+62.924844519"
Sep 10 17:42:19.380412 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:19.380379    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kgrgd"
Sep 10 17:42:21.701534 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:21.701494    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:42:21.704915 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:21.704887    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 17:42:21.714930 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:21.714899    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8cae49c8-331a-409b-8f6a-7d096c469ca5-original-pull-secret\") pod \"global-pull-secret-syncer-mj85b\" (UID: \"8cae49c8-331a-409b-8f6a-7d096c469ca5\") " pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:42:21.802452 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:21.802353    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs\") pod \"network-metrics-daemon-tnrvx\" (UID: \"78bdfa50-029c-4951-9ca4-4bfa31adf9a8\") " pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:42:21.808271 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:21.808239    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 17:42:21.816005 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:21.815973    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78bdfa50-029c-4951-9ca4-4bfa31adf9a8-metrics-certs\") pod \"network-metrics-daemon-tnrvx\" (UID: \"78bdfa50-029c-4951-9ca4-4bfa31adf9a8\") " pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:42:21.903762 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:21.903651    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjtkq\" (UniqueName: \"kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq\") pod \"network-check-target-5q47l\" (UID: \"d61add86-4475-4f8f-b6d4-b3b298df34f4\") " pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:42:21.906874 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:21.906847    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 17:42:21.917173 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:21.917145    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 17:42:21.928856 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:21.928798    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjtkq\" (UniqueName: \"kubernetes.io/projected/d61add86-4475-4f8f-b6d4-b3b298df34f4-kube-api-access-pjtkq\") pod \"network-check-target-5q47l\" (UID: \"d61add86-4475-4f8f-b6d4-b3b298df34f4\") " pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:42:21.932365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:21.932335    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mj85b"
Sep 10 17:42:21.941568 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:21.941535    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hkln9\""
Sep 10 17:42:21.945065 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:21.945043    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fbgqn\""
Sep 10 17:42:21.949325 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:21.949306    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tnrvx"
Sep 10 17:42:21.952674 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:21.952567    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:42:24.051284 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:24.051119    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tnrvx"]
Sep 10 17:42:24.051284 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:24.051192    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mj85b"]
Sep 10 17:42:24.052873 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:24.052834    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5q47l"]
Sep 10 17:42:24.058968 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:42:24.058943    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cae49c8_331a_409b_8f6a_7d096c469ca5.slice/crio-9be7337ec4d2c811e718f2f80e89a942840a774d2420c9096624b412aecc03ba WatchSource:0}: Error finding container 9be7337ec4d2c811e718f2f80e89a942840a774d2420c9096624b412aecc03ba: Status 404 returned error can't find the container with id 9be7337ec4d2c811e718f2f80e89a942840a774d2420c9096624b412aecc03ba
Sep 10 17:42:24.060822 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:42:24.060799    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd61add86_4475_4f8f_b6d4_b3b298df34f4.slice/crio-dde4de426e9e86576317b6f28c0a5be3ff021af8501df169317b9751270421d1 WatchSource:0}: Error finding container dde4de426e9e86576317b6f28c0a5be3ff021af8501df169317b9751270421d1: Status 404 returned error can't find the container with id dde4de426e9e86576317b6f28c0a5be3ff021af8501df169317b9751270421d1
Sep 10 17:42:24.061615 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:42:24.061556    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78bdfa50_029c_4951_9ca4_4bfa31adf9a8.slice/crio-54d43d82c5d5606d6d5bcfd671dc1651d7de169c86cececbe9fc1d54b2ac3571 WatchSource:0}: Error finding container 54d43d82c5d5606d6d5bcfd671dc1651d7de169c86cececbe9fc1d54b2ac3571: Status 404 returned error can't find the container with id 54d43d82c5d5606d6d5bcfd671dc1651d7de169c86cececbe9fc1d54b2ac3571
Sep 10 17:42:24.442398 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:24.442301    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tnrvx" event={"ID":"78bdfa50-029c-4951-9ca4-4bfa31adf9a8","Type":"ContainerStarted","Data":"54d43d82c5d5606d6d5bcfd671dc1651d7de169c86cececbe9fc1d54b2ac3571"}
Sep 10 17:42:24.443704 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:24.443665    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mj85b" event={"ID":"8cae49c8-331a-409b-8f6a-7d096c469ca5","Type":"ContainerStarted","Data":"9be7337ec4d2c811e718f2f80e89a942840a774d2420c9096624b412aecc03ba"}
Sep 10 17:42:24.444977 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:24.444948    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5q47l" event={"ID":"d61add86-4475-4f8f-b6d4-b3b298df34f4","Type":"ContainerStarted","Data":"dde4de426e9e86576317b6f28c0a5be3ff021af8501df169317b9751270421d1"}
Sep 10 17:42:24.446930 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:24.446720    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-558b47745b-m8cc6" event={"ID":"0c62b3e5-82e1-4608-bfaa-815ca05f9bee","Type":"ContainerStarted","Data":"0d067ccfabc168d2150c152ef14b592f7d044fcd60a7a0c9cac99aacd087111d"}
Sep 10 17:42:24.455020 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:24.454979    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf" event={"ID":"7aee70cf-530b-45b8-bd3f-819391da6d1e","Type":"ContainerStarted","Data":"3895ce8bbb8e5271ef4b1081d999de74283340b7a8268908428ec7d1ba75a5af"}
Sep 10 17:42:24.455234 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:24.455210    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"
Sep 10 17:42:24.457178 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:24.457155    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf"
Sep 10 17:42:24.468750 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:24.468657    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-558b47745b-m8cc6" podStartSLOduration=1.839391126 podStartE2EDuration="7.468642839s" totalImagesPullingTime="5.629251713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:24.467573979 +0000 UTC m=+68.958625788" watchObservedRunningTime="2026-09-10 17:42:24.468642839 +0000 UTC m=+68.959694646"
Sep 10 17:42:25.464839 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:25.464759    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tnrvx" event={"ID":"78bdfa50-029c-4951-9ca4-4bfa31adf9a8","Type":"ContainerStarted","Data":"b3698b65228b3440748f157e24b7c303912e41776ad853e4519042c8ba079d0c"}
Sep 10 17:42:25.464839 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:25.464824    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tnrvx" event={"ID":"78bdfa50-029c-4951-9ca4-4bfa31adf9a8","Type":"ContainerStarted","Data":"7537b64f5b38fdd21f8f1dae751a6720eabcaae7d8febd7f5a0d71a2b09db84e"}
Sep 10 17:42:25.485597 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:25.485065    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tnrvx" podStartSLOduration=68.3342063 podStartE2EDuration="1m9.485042861s" totalImagesPullingTime="1.150836561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:25.483377101 +0000 UTC m=+69.974428935" watchObservedRunningTime="2026-09-10 17:42:25.485042861 +0000 UTC m=+69.976094671"
Sep 10 17:42:25.485597 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:25.485568    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dd8798dd4-6mdcf" podStartSLOduration=2.8113532 podStartE2EDuration="8.485555873s" totalImagesPullingTime="5.674202673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:24.486022839 +0000 UTC m=+68.977074651" watchObservedRunningTime="2026-09-10 17:42:25.485555873 +0000 UTC m=+69.976607683"
Sep 10 17:42:25.636905 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:25.635621    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-jrgfx"]
Sep 10 17:42:25.638204 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:25.638180    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-jrgfx"
Sep 10 17:42:25.641039 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:25.641013    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-rdblf\""
Sep 10 17:42:25.641346 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:25.641324    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 10 17:42:25.651411 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:25.651389    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-jrgfx"]
Sep 10 17:42:25.733961 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:25.733877    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/65998d10-dfba-422e-bf2e-692ca983f800-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-jrgfx\" (UID: \"65998d10-dfba-422e-bf2e-692ca983f800\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-jrgfx"
Sep 10 17:42:25.834460 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:25.834419    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/65998d10-dfba-422e-bf2e-692ca983f800-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-jrgfx\" (UID: \"65998d10-dfba-422e-bf2e-692ca983f800\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-jrgfx"
Sep 10 17:42:25.834630 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:42:25.834583    2549 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 10 17:42:25.834680 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:42:25.834663    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65998d10-dfba-422e-bf2e-692ca983f800-tls-certificates podName:65998d10-dfba-422e-bf2e-692ca983f800 nodeName:}" failed. No retries permitted until 2026-09-10 17:42:26.334638962 +0000 UTC m=+70.825690773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/65998d10-dfba-422e-bf2e-692ca983f800-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-jrgfx" (UID: "65998d10-dfba-422e-bf2e-692ca983f800") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 10 17:42:26.339248 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:26.339206    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/65998d10-dfba-422e-bf2e-692ca983f800-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-jrgfx\" (UID: \"65998d10-dfba-422e-bf2e-692ca983f800\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-jrgfx"
Sep 10 17:42:26.342506 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:26.342475    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/65998d10-dfba-422e-bf2e-692ca983f800-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-jrgfx\" (UID: \"65998d10-dfba-422e-bf2e-692ca983f800\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-jrgfx"
Sep 10 17:42:26.552408 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:26.552310    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-jrgfx"
Sep 10 17:42:26.841202 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:26.841165    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-jrgfx"]
Sep 10 17:42:27.477833 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:27.477795    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-jrgfx" event={"ID":"65998d10-dfba-422e-bf2e-692ca983f800","Type":"ContainerStarted","Data":"c95041b227fda553f25d7a677cb2f4e3f80a1f62cddb75fd5a34fbfe28ec2125"}
Sep 10 17:42:37.520961 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:37.520921    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mj85b" event={"ID":"8cae49c8-331a-409b-8f6a-7d096c469ca5","Type":"ContainerStarted","Data":"6ce11ea6a8992bfa61487a61cfe2b15242bf9dc75fd422cd256b32fe04febc6d"}
Sep 10 17:42:37.522449 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:37.522421    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5q47l" event={"ID":"d61add86-4475-4f8f-b6d4-b3b298df34f4","Type":"ContainerStarted","Data":"864f0daff94fa259636d5d81046880a90c2807ae7dc4fc20e8282c78adcee450"}
Sep 10 17:42:37.522561 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:37.522544    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:42:37.538808 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:37.538756    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mj85b" podStartSLOduration=68.872988506 podStartE2EDuration="1m21.538721911s" totalImagesPullingTime="12.665733405s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:37.53742287 +0000 UTC m=+82.028474678" watchObservedRunningTime="2026-09-10 17:42:37.538721911 +0000 UTC m=+82.029773713"
Sep 10 17:42:37.554382 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:37.554334    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5q47l" podStartSLOduration=68.891570158 podStartE2EDuration="1m21.554318728s" totalImagesPullingTime="12.66274857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:37.553556989 +0000 UTC m=+82.044608811" watchObservedRunningTime="2026-09-10 17:42:37.554318728 +0000 UTC m=+82.045370536"
Sep 10 17:42:38.528607 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.528429    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-4nnsl" event={"ID":"b459cafb-dae0-4cfd-9859-ed15e033dc5c","Type":"ContainerStarted","Data":"c08a5ef325d267d7de095baf4c991a73d62a90711228fd8bd1d2121f5f1939e4"}
Sep 10 17:42:38.528607 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.528500    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-4nnsl"
Sep 10 17:42:38.530503 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.530476    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-jrgfx" event={"ID":"65998d10-dfba-422e-bf2e-692ca983f800","Type":"ContainerStarted","Data":"5f570bff8626b0f63370f0efe7e27133b9ae602893fe98215996be7011fad66a"}
Sep 10 17:42:38.531170 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.531150    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-jrgfx"
Sep 10 17:42:38.536070 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.536050    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-jrgfx"
Sep 10 17:42:38.547839 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.547788    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-4nnsl" podStartSLOduration=2.160160086 podStartE2EDuration="21.547770338s" totalImagesPullingTime="19.387610252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:38.54751471 +0000 UTC m=+83.038566522" watchObservedRunningTime="2026-09-10 17:42:38.547770338 +0000 UTC m=+83.038822150"
Sep 10 17:42:38.549800 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.549770    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-4nnsl"
Sep 10 17:42:38.562616 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.562515    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-jrgfx" podStartSLOduration=2.825227896 podStartE2EDuration="13.562495642s" totalImagesPullingTime="10.737267746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:38.562164125 +0000 UTC m=+83.053215934" watchObservedRunningTime="2026-09-10 17:42:38.562495642 +0000 UTC m=+83.053547450"
Sep 10 17:42:38.699582 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.699531    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"]
Sep 10 17:42:38.714115 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.714080    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"]
Sep 10 17:42:38.714313 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.714222    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"
Sep 10 17:42:38.717199 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.717170    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 17:42:38.717553 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.717527    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-fzndc\""
Sep 10 17:42:38.717915 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.717892    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 10 17:42:38.718024 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.717965    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 17:42:38.718099 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.717797    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 17:42:38.718164 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.717857    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 10 17:42:38.831102 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.831005    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfv2l\" (UniqueName: \"kubernetes.io/projected/b77d1245-6e62-4c4d-afdf-471974a7fa89-kube-api-access-lfv2l\") pod \"prometheus-operator-6f4946db56-gcdnv\" (UID: \"b77d1245-6e62-4c4d-afdf-471974a7fa89\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"
Sep 10 17:42:38.831102 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.831051    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b77d1245-6e62-4c4d-afdf-471974a7fa89-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-gcdnv\" (UID: \"b77d1245-6e62-4c4d-afdf-471974a7fa89\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"
Sep 10 17:42:38.831325 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.831137    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b77d1245-6e62-4c4d-afdf-471974a7fa89-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-gcdnv\" (UID: \"b77d1245-6e62-4c4d-afdf-471974a7fa89\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"
Sep 10 17:42:38.831325 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.831197    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b77d1245-6e62-4c4d-afdf-471974a7fa89-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-gcdnv\" (UID: \"b77d1245-6e62-4c4d-afdf-471974a7fa89\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"
Sep 10 17:42:38.931900 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.931861    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b77d1245-6e62-4c4d-afdf-471974a7fa89-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-gcdnv\" (UID: \"b77d1245-6e62-4c4d-afdf-471974a7fa89\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"
Sep 10 17:42:38.932093 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.931945    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b77d1245-6e62-4c4d-afdf-471974a7fa89-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-gcdnv\" (UID: \"b77d1245-6e62-4c4d-afdf-471974a7fa89\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"
Sep 10 17:42:38.932093 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.931976    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfv2l\" (UniqueName: \"kubernetes.io/projected/b77d1245-6e62-4c4d-afdf-471974a7fa89-kube-api-access-lfv2l\") pod \"prometheus-operator-6f4946db56-gcdnv\" (UID: \"b77d1245-6e62-4c4d-afdf-471974a7fa89\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"
Sep 10 17:42:38.932093 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.932003    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b77d1245-6e62-4c4d-afdf-471974a7fa89-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-gcdnv\" (UID: \"b77d1245-6e62-4c4d-afdf-471974a7fa89\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"
Sep 10 17:42:38.932739 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.932684    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b77d1245-6e62-4c4d-afdf-471974a7fa89-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-gcdnv\" (UID: \"b77d1245-6e62-4c4d-afdf-471974a7fa89\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"
Sep 10 17:42:38.934936 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.934911    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b77d1245-6e62-4c4d-afdf-471974a7fa89-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-gcdnv\" (UID: \"b77d1245-6e62-4c4d-afdf-471974a7fa89\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"
Sep 10 17:42:38.935056 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.935042    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b77d1245-6e62-4c4d-afdf-471974a7fa89-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-gcdnv\" (UID: \"b77d1245-6e62-4c4d-afdf-471974a7fa89\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"
Sep 10 17:42:38.941077 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:38.940950    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfv2l\" (UniqueName: \"kubernetes.io/projected/b77d1245-6e62-4c4d-afdf-471974a7fa89-kube-api-access-lfv2l\") pod \"prometheus-operator-6f4946db56-gcdnv\" (UID: \"b77d1245-6e62-4c4d-afdf-471974a7fa89\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"
Sep 10 17:42:39.026082 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:39.026039    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"
Sep 10 17:42:39.337681 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:39.335533    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-gcdnv"]
Sep 10 17:42:39.424752 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:39.424655    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-58d4769c69-rjtx9"
Sep 10 17:42:39.536546 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:39.536507    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv" event={"ID":"b77d1245-6e62-4c4d-afdf-471974a7fa89","Type":"ContainerStarted","Data":"0d2771d93f437275ae477eabf2ad2da95b77ac31bce0b076741fb1dd37f65a15"}
Sep 10 17:42:41.548286 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:41.548205    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv" event={"ID":"b77d1245-6e62-4c4d-afdf-471974a7fa89","Type":"ContainerStarted","Data":"40057f55c491da2f4357700b4f25c7460caa1cac3bbab381cd04efb1216e121d"}
Sep 10 17:42:41.548286 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:41.548270    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv" event={"ID":"b77d1245-6e62-4c4d-afdf-471974a7fa89","Type":"ContainerStarted","Data":"5b6dbdcc22bbc587f8cab2f11c3092a6443518ba5d33a3d64a350b1f1c551102"}
Sep 10 17:42:41.567391 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:41.567338    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-gcdnv" podStartSLOduration=1.701188588 podStartE2EDuration="3.567315241s" totalImagesPullingTime="1.866126653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:41.565775579 +0000 UTC m=+86.056827395" watchObservedRunningTime="2026-09-10 17:42:41.567315241 +0000 UTC m=+86.058367050"
Sep 10 17:42:43.081287 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.081246    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dbzh4"]
Sep 10 17:42:43.104642 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.104612    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.107985 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.107808    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 17:42:43.107985 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.107826    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 17:42:43.107985 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.107847    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 17:42:43.107985 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.107849    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5vtvp\""
Sep 10 17:42:43.167504 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.167470    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/baf11260-22e8-4976-91f0-c417469342da-node-exporter-accelerators-collector-config\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.167680 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.167511    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/baf11260-22e8-4976-91f0-c417469342da-node-exporter-textfile\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.167680 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.167543    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95dql\" (UniqueName: \"kubernetes.io/projected/baf11260-22e8-4976-91f0-c417469342da-kube-api-access-95dql\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.167680 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.167632    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/baf11260-22e8-4976-91f0-c417469342da-node-exporter-wtmp\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.167680 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.167667    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/baf11260-22e8-4976-91f0-c417469342da-root\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.167908 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.167686    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/baf11260-22e8-4976-91f0-c417469342da-node-exporter-tls\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.167908 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.167768    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/baf11260-22e8-4976-91f0-c417469342da-sys\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.167908 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.167806    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/baf11260-22e8-4976-91f0-c417469342da-metrics-client-ca\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.167908 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.167834    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/baf11260-22e8-4976-91f0-c417469342da-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.269096 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.269057    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/baf11260-22e8-4976-91f0-c417469342da-node-exporter-wtmp\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.269286 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.269106    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/baf11260-22e8-4976-91f0-c417469342da-root\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.269286 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.269129    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/baf11260-22e8-4976-91f0-c417469342da-node-exporter-tls\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.269286 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.269157    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/baf11260-22e8-4976-91f0-c417469342da-sys\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.269286 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.269182    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/baf11260-22e8-4976-91f0-c417469342da-metrics-client-ca\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.269286 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.269204    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/baf11260-22e8-4976-91f0-c417469342da-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.269286 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.269201    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/baf11260-22e8-4976-91f0-c417469342da-root\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.269286 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.269250    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/baf11260-22e8-4976-91f0-c417469342da-node-exporter-wtmp\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.269286 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.269263    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/baf11260-22e8-4976-91f0-c417469342da-sys\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.269286 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.269275    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/baf11260-22e8-4976-91f0-c417469342da-node-exporter-accelerators-collector-config\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.269751 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.269307    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/baf11260-22e8-4976-91f0-c417469342da-node-exporter-textfile\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.269751 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.269332    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95dql\" (UniqueName: \"kubernetes.io/projected/baf11260-22e8-4976-91f0-c417469342da-kube-api-access-95dql\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.269966 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.269941    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/baf11260-22e8-4976-91f0-c417469342da-metrics-client-ca\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.270174 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.270140    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/baf11260-22e8-4976-91f0-c417469342da-node-exporter-textfile\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.270522 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.270498    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/baf11260-22e8-4976-91f0-c417469342da-node-exporter-accelerators-collector-config\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.271958 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.271934    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/baf11260-22e8-4976-91f0-c417469342da-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.272062 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.272035    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/baf11260-22e8-4976-91f0-c417469342da-node-exporter-tls\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.278677 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.278658    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95dql\" (UniqueName: \"kubernetes.io/projected/baf11260-22e8-4976-91f0-c417469342da-kube-api-access-95dql\") pod \"node-exporter-dbzh4\" (UID: \"baf11260-22e8-4976-91f0-c417469342da\") " pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.415903 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.415819    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dbzh4"
Sep 10 17:42:43.452651 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:42:43.452604    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaf11260_22e8_4976_91f0_c417469342da.slice/crio-6a4ba924fe709c7153d01a846aa5141b1c3409e34535d56b6b028ecc7fb98b3d WatchSource:0}: Error finding container 6a4ba924fe709c7153d01a846aa5141b1c3409e34535d56b6b028ecc7fb98b3d: Status 404 returned error can't find the container with id 6a4ba924fe709c7153d01a846aa5141b1c3409e34535d56b6b028ecc7fb98b3d
Sep 10 17:42:43.559137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:43.559095    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dbzh4" event={"ID":"baf11260-22e8-4976-91f0-c417469342da","Type":"ContainerStarted","Data":"6a4ba924fe709c7153d01a846aa5141b1c3409e34535d56b6b028ecc7fb98b3d"}
Sep 10 17:42:45.568240 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:45.568202    2549 generic.go:356] "Generic (PLEG): container finished" podID="baf11260-22e8-4976-91f0-c417469342da" containerID="0ca3cf84b518d3e062aaf3db5a2ff8e52315021c6b1c3ea97dd8558f3e2e5a68" exitCode=0
Sep 10 17:42:45.568766 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:45.568301    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dbzh4" event={"ID":"baf11260-22e8-4976-91f0-c417469342da","Type":"ContainerDied","Data":"0ca3cf84b518d3e062aaf3db5a2ff8e52315021c6b1c3ea97dd8558f3e2e5a68"}
Sep 10 17:42:46.031857 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.031826    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"]
Sep 10 17:42:46.053335 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.053304    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"]
Sep 10 17:42:46.053526 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.053446    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.057299 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.057273    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 10 17:42:46.057453 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.057435    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 10 17:42:46.057542 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.057526    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 10 17:42:46.057637 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.057620    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 10 17:42:46.057750 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.057658    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 10 17:42:46.057863 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.057669    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2cgkp\""
Sep 10 17:42:46.057863 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.057778    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-39qvhn38bhtrh\""
Sep 10 17:42:46.195703 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.195659    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.195873 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.195725    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.195873 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.195753    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-grpc-tls\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.195873 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.195777    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.195873 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.195804    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.195873 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.195852    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbnwm\" (UniqueName: \"kubernetes.io/projected/3901e250-2092-4cde-9ef7-d3602bde097e-kube-api-access-nbnwm\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.196100 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.195879    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3901e250-2092-4cde-9ef7-d3602bde097e-metrics-client-ca\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.196100 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.196003    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-thanos-querier-tls\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.297235 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.297195    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-thanos-querier-tls\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.297425 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.297243    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.297425 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.297291    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.297425 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.297316    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-grpc-tls\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.297425 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.297341    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.297425 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.297373    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.297425 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.297400    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbnwm\" (UniqueName: \"kubernetes.io/projected/3901e250-2092-4cde-9ef7-d3602bde097e-kube-api-access-nbnwm\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.297425 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.297426    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3901e250-2092-4cde-9ef7-d3602bde097e-metrics-client-ca\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.298980 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.298945    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3901e250-2092-4cde-9ef7-d3602bde097e-metrics-client-ca\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.300493 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.300466    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.302449 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.301298    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.302449 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.301542    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.302449 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.301568    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-thanos-querier-tls\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.302449 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.302038    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.303537 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.303516    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3901e250-2092-4cde-9ef7-d3602bde097e-secret-grpc-tls\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.309432 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.309408    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbnwm\" (UniqueName: \"kubernetes.io/projected/3901e250-2092-4cde-9ef7-d3602bde097e-kube-api-access-nbnwm\") pod \"thanos-querier-5dd6ffb6df-m97z6\" (UID: \"3901e250-2092-4cde-9ef7-d3602bde097e\") " pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.365787 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.365755    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:46.575195 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.575157    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dbzh4" event={"ID":"baf11260-22e8-4976-91f0-c417469342da","Type":"ContainerStarted","Data":"dc534f4f17594715b5d07e70f2fa4ad3ca67a91bd757a828c8514ee069284a0c"}
Sep 10 17:42:46.575617 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.575215    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dbzh4" event={"ID":"baf11260-22e8-4976-91f0-c417469342da","Type":"ContainerStarted","Data":"d37aa54a235fbf10b3d2dcec0192243ea3643b49bdc3d4c7a87e4f2573e0dd80"}
Sep 10 17:42:46.602007 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.598892    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dbzh4" podStartSLOduration=1.875067154 podStartE2EDuration="3.598872099s" totalImagesPullingTime="1.641183883s" totalInitContainerRuntime="82.621062ms" isStatefulPod=true podCreationTimestamp="2026-09-10 17:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:46.597123312 +0000 UTC m=+91.088175124" watchObservedRunningTime="2026-09-10 17:42:46.598872099 +0000 UTC m=+91.089923909"
Sep 10 17:42:46.686193 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:46.686149    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"]
Sep 10 17:42:46.689889 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:42:46.689856    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3901e250_2092_4cde_9ef7_d3602bde097e.slice/crio-49f112bfa66c82d4281c84448f28330462d5cd225eafeedae24a989b2ce586c2 WatchSource:0}: Error finding container 49f112bfa66c82d4281c84448f28330462d5cd225eafeedae24a989b2ce586c2: Status 404 returned error can't find the container with id 49f112bfa66c82d4281c84448f28330462d5cd225eafeedae24a989b2ce586c2
Sep 10 17:42:47.450002 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.449961    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-8468bdbf64-f754m"]
Sep 10 17:42:47.476794 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.476757    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8468bdbf64-f754m"]
Sep 10 17:42:47.476956 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.476862    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.483707 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.483665    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 17:42:47.483873 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.483715    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 10 17:42:47.485072 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.485052    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 10 17:42:47.485193 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.485052    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-vs5sc\""
Sep 10 17:42:47.485193 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.485096    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-b9nj6mo3nc9ht\""
Sep 10 17:42:47.485193 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.485104    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 10 17:42:47.581064 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.580987    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6" event={"ID":"3901e250-2092-4cde-9ef7-d3602bde097e","Type":"ContainerStarted","Data":"49f112bfa66c82d4281c84448f28330462d5cd225eafeedae24a989b2ce586c2"}
Sep 10 17:42:47.610754 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.610720    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f637a746-636f-404c-9593-3e8b393b1663-client-ca-bundle\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.610914 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.610777    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nh8d\" (UniqueName: \"kubernetes.io/projected/f637a746-636f-404c-9593-3e8b393b1663-kube-api-access-2nh8d\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.610914 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.610808    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f637a746-636f-404c-9593-3e8b393b1663-secret-metrics-server-client-certs\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.610914 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.610834    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f637a746-636f-404c-9593-3e8b393b1663-audit-log\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.611070 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.610931    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f637a746-636f-404c-9593-3e8b393b1663-secret-metrics-server-tls\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.611070 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.610982    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f637a746-636f-404c-9593-3e8b393b1663-metrics-server-audit-profiles\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.611070 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.611054    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f637a746-636f-404c-9593-3e8b393b1663-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.711786 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.711678    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f637a746-636f-404c-9593-3e8b393b1663-audit-log\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.711955 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.711888    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f637a746-636f-404c-9593-3e8b393b1663-secret-metrics-server-tls\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.712022 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.711968    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f637a746-636f-404c-9593-3e8b393b1663-metrics-server-audit-profiles\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.712062 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.712048    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f637a746-636f-404c-9593-3e8b393b1663-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.712158 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.712139    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f637a746-636f-404c-9593-3e8b393b1663-audit-log\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.712468 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.712247    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f637a746-636f-404c-9593-3e8b393b1663-client-ca-bundle\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.712468 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.712455    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nh8d\" (UniqueName: \"kubernetes.io/projected/f637a746-636f-404c-9593-3e8b393b1663-kube-api-access-2nh8d\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.712647 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.712513    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f637a746-636f-404c-9593-3e8b393b1663-secret-metrics-server-client-certs\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.712778 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.712733    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f637a746-636f-404c-9593-3e8b393b1663-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.713146 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.713102    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f637a746-636f-404c-9593-3e8b393b1663-metrics-server-audit-profiles\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.715190 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.715166    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f637a746-636f-404c-9593-3e8b393b1663-secret-metrics-server-client-certs\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.715292 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.715245    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f637a746-636f-404c-9593-3e8b393b1663-secret-metrics-server-tls\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.715340 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.715311    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f637a746-636f-404c-9593-3e8b393b1663-client-ca-bundle\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.720659 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.720633    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nh8d\" (UniqueName: \"kubernetes.io/projected/f637a746-636f-404c-9593-3e8b393b1663-kube-api-access-2nh8d\") pod \"metrics-server-8468bdbf64-f754m\" (UID: \"f637a746-636f-404c-9593-3e8b393b1663\") " pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:47.786091 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:47.786057    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:42:48.107157 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:48.107052    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8468bdbf64-f754m"]
Sep 10 17:42:48.677152 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:42:48.677117    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf637a746_636f_404c_9593_3e8b393b1663.slice/crio-bf32f1335f7a3f773629d4c814bb08d0bd475fdcd1b4f7f2e9982369fa133b6b WatchSource:0}: Error finding container bf32f1335f7a3f773629d4c814bb08d0bd475fdcd1b4f7f2e9982369fa133b6b: Status 404 returned error can't find the container with id bf32f1335f7a3f773629d4c814bb08d0bd475fdcd1b4f7f2e9982369fa133b6b
Sep 10 17:42:49.288791 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.288756    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:42:49.307443 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.307411    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.317780 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.315135    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f6eegfg36d3q5\""
Sep 10 17:42:49.317780 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.315261    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 17:42:49.317780 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.315471    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 17:42:49.317780 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.315679    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 17:42:49.317780 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.315921    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 17:42:49.317780 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.316145    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 17:42:49.317780 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.316335    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 17:42:49.317780 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.316633    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ltc8f\""
Sep 10 17:42:49.317780 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.316978    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 17:42:49.317780 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.317096    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 17:42:49.317780 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.317199    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 17:42:49.317780 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.317275    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 17:42:49.317780 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.317682    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:42:49.318645 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.318592    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 17:42:49.319334 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.318742    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 17:42:49.326583 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.326555    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 17:42:49.334259 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.334189    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 17:42:49.429962 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.429923    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430142 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.429972    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430142 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430005    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjc8v\" (UniqueName: \"kubernetes.io/projected/5960e955-5d77-448d-9f32-b0c25a5e71de-kube-api-access-fjc8v\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430142 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430031    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430142 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430061    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430160    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5960e955-5d77-448d-9f32-b0c25a5e71de-config-out\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430183    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430215    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430249    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-config\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430274    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430311    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430346    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430372    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430396    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430779 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430418    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430779 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430468    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-web-config\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430779 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430519    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5960e955-5d77-448d-9f32-b0c25a5e71de-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430779 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430551    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430779 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430579    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.430779 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.430625    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.531956 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.531919    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532135 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.531965    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532135 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.531988    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532135 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532009    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532135 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532030    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-web-config\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532135 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532056    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5960e955-5d77-448d-9f32-b0c25a5e71de-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532135 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532086    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532135 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532114    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532145    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532195    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532219    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532252    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjc8v\" (UniqueName: \"kubernetes.io/projected/5960e955-5d77-448d-9f32-b0c25a5e71de-kube-api-access-fjc8v\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532277    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532311    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532344    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5960e955-5d77-448d-9f32-b0c25a5e71de-config-out\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532370    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532375    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532414    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532446    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-config\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532470    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.532499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.532503    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.533406 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.533379    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.534499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.534470    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.534820 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.534796    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.535016 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.534993    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.535687 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.535658    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.537832 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.536047    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.538768 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.538687    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.539804 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.539760    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.540707 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.540650    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.541260 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.541237    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.541892 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.541868    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.542491 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.542328    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.542491 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.542457    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.542640 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.542611    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-web-config\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.542771 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.542754    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.542841 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.542801    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-config\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.542955 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.542936    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5960e955-5d77-448d-9f32-b0c25a5e71de-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.543782 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.543760    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5960e955-5d77-448d-9f32-b0c25a5e71de-config-out\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.544364 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.544347    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjc8v\" (UniqueName: \"kubernetes.io/projected/5960e955-5d77-448d-9f32-b0c25a5e71de-kube-api-access-fjc8v\") pod \"prometheus-k8s-0\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.597759 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.597657    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6" event={"ID":"3901e250-2092-4cde-9ef7-d3602bde097e","Type":"ContainerStarted","Data":"1add0856f735f65f51912545b5092795b06f837c4c6cf1eb0ece0dbdb0c77812"}
Sep 10 17:42:49.597916 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.597815    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6" event={"ID":"3901e250-2092-4cde-9ef7-d3602bde097e","Type":"ContainerStarted","Data":"233587663b9795a9bfa5a09a197912e6dbee133e0c5e19f712fba938706b0040"}
Sep 10 17:42:49.597916 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.597859    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6" event={"ID":"3901e250-2092-4cde-9ef7-d3602bde097e","Type":"ContainerStarted","Data":"633414250ae8aa81088419ebe20449f6dd0d612476dd4ae1025212783bfe375e"}
Sep 10 17:42:49.599118 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.599088    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8468bdbf64-f754m" event={"ID":"f637a746-636f-404c-9593-3e8b393b1663","Type":"ContainerStarted","Data":"bf32f1335f7a3f773629d4c814bb08d0bd475fdcd1b4f7f2e9982369fa133b6b"}
Sep 10 17:42:49.633391 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.633280    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:42:49.953662 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:49.953632    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:42:50.208223 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:42:50.208145    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5960e955_5d77_448d_9f32_b0c25a5e71de.slice/crio-dced9a0c13656216c1d957698577fce8633cb4570e2580812f2bcc6f381d2128 WatchSource:0}: Error finding container dced9a0c13656216c1d957698577fce8633cb4570e2580812f2bcc6f381d2128: Status 404 returned error can't find the container with id dced9a0c13656216c1d957698577fce8633cb4570e2580812f2bcc6f381d2128
Sep 10 17:42:50.604420 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:50.604382    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5960e955-5d77-448d-9f32-b0c25a5e71de","Type":"ContainerStarted","Data":"dced9a0c13656216c1d957698577fce8633cb4570e2580812f2bcc6f381d2128"}
Sep 10 17:42:51.609916 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:51.609879    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8468bdbf64-f754m" event={"ID":"f637a746-636f-404c-9593-3e8b393b1663","Type":"ContainerStarted","Data":"c33d3d8da53ba647609a81c3200363ec7ab00df0be9c8df6359eb6ac858e0fa0"}
Sep 10 17:42:51.631063 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:51.631017    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-8468bdbf64-f754m" podStartSLOduration=2.768700743 podStartE2EDuration="4.631001629s" totalImagesPullingTime="1.862300886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:51.6304061 +0000 UTC m=+96.121457911" watchObservedRunningTime="2026-09-10 17:42:51.631001629 +0000 UTC m=+96.122053437"
Sep 10 17:42:52.617372 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:52.617336    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6" event={"ID":"3901e250-2092-4cde-9ef7-d3602bde097e","Type":"ContainerStarted","Data":"ba8e47b56b8cce6929b8862943ff73ac74d5eefecdc0daed63c75ffd6360165b"}
Sep 10 17:42:52.617815 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:52.617403    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6" event={"ID":"3901e250-2092-4cde-9ef7-d3602bde097e","Type":"ContainerStarted","Data":"12ea5eb9f512591c95e1909d1ae3285c947dabbe9e4c266661610e5b3e211d6c"}
Sep 10 17:42:52.617815 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:52.617439    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6" event={"ID":"3901e250-2092-4cde-9ef7-d3602bde097e","Type":"ContainerStarted","Data":"abda0a0ab56c200fb34e230ad85f6d9f36c5e8adcfd7d266d5ed72a94efc9464"}
Sep 10 17:42:52.617815 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:52.617512    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:52.619041 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:52.619019    2549 generic.go:356] "Generic (PLEG): container finished" podID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerID="fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3" exitCode=0
Sep 10 17:42:52.619122 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:52.619085    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5960e955-5d77-448d-9f32-b0c25a5e71de","Type":"ContainerDied","Data":"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3"}
Sep 10 17:42:52.677455 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:52.677399    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6" podStartSLOduration=2.105771791 podStartE2EDuration="6.677380989s" totalImagesPullingTime="4.571609198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:52.642602416 +0000 UTC m=+97.133654253" watchObservedRunningTime="2026-09-10 17:42:52.677380989 +0000 UTC m=+97.168432829"
Sep 10 17:42:56.639818 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:56.639790    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5960e955-5d77-448d-9f32-b0c25a5e71de","Type":"ContainerStarted","Data":"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"}
Sep 10 17:42:56.640105 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:56.639853    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5960e955-5d77-448d-9f32-b0c25a5e71de","Type":"ContainerStarted","Data":"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"}
Sep 10 17:42:56.640105 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:56.639880    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5960e955-5d77-448d-9f32-b0c25a5e71de","Type":"ContainerStarted","Data":"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"}
Sep 10 17:42:56.640105 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:56.639905    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5960e955-5d77-448d-9f32-b0c25a5e71de","Type":"ContainerStarted","Data":"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"}
Sep 10 17:42:57.647092 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:57.647055    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5960e955-5d77-448d-9f32-b0c25a5e71de","Type":"ContainerStarted","Data":"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"}
Sep 10 17:42:57.647662 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:57.647115    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5960e955-5d77-448d-9f32-b0c25a5e71de","Type":"ContainerStarted","Data":"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"}
Sep 10 17:42:57.680581 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:57.680524    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.661630751 podStartE2EDuration="8.680505307s" totalImagesPullingTime="5.000169934s" totalInitContainerRuntime="18.704622ms" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:57.677548945 +0000 UTC m=+102.168600758" watchObservedRunningTime="2026-09-10 17:42:57.680505307 +0000 UTC m=+102.171557115"
Sep 10 17:42:58.630050 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:58.630018    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5dd6ffb6df-m97z6"
Sep 10 17:42:59.634535 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:42:59.634495    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:43:07.786812 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:43:07.786771    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:43:07.787484 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:43:07.786860    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:43:08.543174 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:43:08.543134    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5q47l"
Sep 10 17:43:27.792059 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:43:27.792029    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:43:27.795823 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:43:27.795797    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-8468bdbf64-f754m"
Sep 10 17:43:49.634137 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:43:49.634095    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:43:49.653243 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:43:49.653212    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:43:49.867551 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:43:49.867525    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:07.633207 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.633170    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:44:07.633648 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.633494    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="kube-rbac-proxy" containerID="cri-o://220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb" gracePeriod=600
Sep 10 17:44:07.633648 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.633536    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="thanos-sidecar" containerID="cri-o://86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc" gracePeriod=600
Sep 10 17:44:07.633648 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.633568    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="kube-rbac-proxy-web" containerID="cri-o://28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe" gracePeriod=600
Sep 10 17:44:07.633648 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.633607    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="config-reloader" containerID="cri-o://07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62" gracePeriod=600
Sep 10 17:44:07.633648 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.633560    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="kube-rbac-proxy-thanos" containerID="cri-o://0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6" gracePeriod=600
Sep 10 17:44:07.633976 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.633499    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="prometheus" containerID="cri-o://8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545" gracePeriod=600
Sep 10 17:44:07.870246 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.870218    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:07.873495 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873476    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.873579 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873508    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjc8v\" (UniqueName: \"kubernetes.io/projected/5960e955-5d77-448d-9f32-b0c25a5e71de-kube-api-access-fjc8v\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.873579 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873526    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-kubelet-serving-ca-bundle\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.873579 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873546    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-serving-certs-ca-bundle\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.873579 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873566    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-db\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.873793 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873601    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-metrics-client-certs\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.873793 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873643    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-kube-rbac-proxy\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.873793 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873670    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-0\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.873793 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873729    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.873793 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873755    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-trusted-ca-bundle\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.873793 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873784    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-tls\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.874104 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873807    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-2\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.874104 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873848    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-config\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.874104 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873876    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5960e955-5d77-448d-9f32-b0c25a5e71de-config-out\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.874104 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873901    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-web-config\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.874104 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873904    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:44:07.874104 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873928    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-thanos-prometheus-http-client-file\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.874104 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873951    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:44:07.874104 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.873983    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-grpc-tls\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.874104 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.874015    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-1\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.874104 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.874051    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5960e955-5d77-448d-9f32-b0c25a5e71de-tls-assets\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.874104 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.874091    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-metrics-client-ca\") pod \"5960e955-5d77-448d-9f32-b0c25a5e71de\" (UID: \"5960e955-5d77-448d-9f32-b0c25a5e71de\") "
Sep 10 17:44:07.874637 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.874345    2549 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.874637 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.874365    2549 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.874781 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.874640    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:44:07.874781 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.874730    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:44:07.875577 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.875545    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:44:07.876038 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.876012    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:44:07.876608 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.876580    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:44:07.876728 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.876654    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:44:07.877080 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.877054    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:44:07.877196 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.877171    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5960e955-5d77-448d-9f32-b0c25a5e71de-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:44:07.878350 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.878322    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-config" (OuterVolumeSpecName: "config") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:44:07.878470 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.878449    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5960e955-5d77-448d-9f32-b0c25a5e71de-config-out" (OuterVolumeSpecName: "config-out") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:44:07.878611 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.878588    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:44:07.879066 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.879027    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:44:07.879378 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.879350    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:44:07.880183 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.880155    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:44:07.880444 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.880414    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:44:07.880733 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.880710    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:44:07.880733 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.880720    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5960e955-5d77-448d-9f32-b0c25a5e71de-kube-api-access-fjc8v" (OuterVolumeSpecName: "kube-api-access-fjc8v") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "kube-api-access-fjc8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:44:07.891040 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.890982    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-web-config" (OuterVolumeSpecName: "web-config") pod "5960e955-5d77-448d-9f32-b0c25a5e71de" (UID: "5960e955-5d77-448d-9f32-b0c25a5e71de"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:44:07.920124 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.920025    2549 generic.go:356] "Generic (PLEG): container finished" podID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerID="0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6" exitCode=0
Sep 10 17:44:07.920124 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.920056    2549 generic.go:356] "Generic (PLEG): container finished" podID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerID="220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb" exitCode=0
Sep 10 17:44:07.920124 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.920065    2549 generic.go:356] "Generic (PLEG): container finished" podID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerID="28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe" exitCode=0
Sep 10 17:44:07.920401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.920364    2549 generic.go:356] "Generic (PLEG): container finished" podID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerID="86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc" exitCode=0
Sep 10 17:44:07.920401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.920378    2549 generic.go:356] "Generic (PLEG): container finished" podID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerID="07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62" exitCode=0
Sep 10 17:44:07.920401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.920385    2549 generic.go:356] "Generic (PLEG): container finished" podID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerID="8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545" exitCode=0
Sep 10 17:44:07.920545 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.920248    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:07.921049 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.920267    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5960e955-5d77-448d-9f32-b0c25a5e71de","Type":"ContainerDied","Data":"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"}
Sep 10 17:44:07.921179 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.921107    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5960e955-5d77-448d-9f32-b0c25a5e71de","Type":"ContainerDied","Data":"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"}
Sep 10 17:44:07.921179 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.921153    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5960e955-5d77-448d-9f32-b0c25a5e71de","Type":"ContainerDied","Data":"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"}
Sep 10 17:44:07.921287 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.921192    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5960e955-5d77-448d-9f32-b0c25a5e71de","Type":"ContainerDied","Data":"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"}
Sep 10 17:44:07.921287 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.921233    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5960e955-5d77-448d-9f32-b0c25a5e71de","Type":"ContainerDied","Data":"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"}
Sep 10 17:44:07.921287 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.921275    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5960e955-5d77-448d-9f32-b0c25a5e71de","Type":"ContainerDied","Data":"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"}
Sep 10 17:44:07.921434 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.921319    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5960e955-5d77-448d-9f32-b0c25a5e71de","Type":"ContainerDied","Data":"dced9a0c13656216c1d957698577fce8633cb4570e2580812f2bcc6f381d2128"}
Sep 10 17:44:07.921434 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.921368    2549 scope.go:117] "RemoveContainer" containerID="0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"
Sep 10 17:44:07.933585 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.933394    2549 scope.go:117] "RemoveContainer" containerID="220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"
Sep 10 17:44:07.945825 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.945800    2549 scope.go:117] "RemoveContainer" containerID="28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"
Sep 10 17:44:07.951909 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.951880    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:44:07.955627 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.955558    2549 scope.go:117] "RemoveContainer" containerID="86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"
Sep 10 17:44:07.955870 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.955850    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:44:07.962984 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.962963    2549 scope.go:117] "RemoveContainer" containerID="07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"
Sep 10 17:44:07.970030 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.970005    2549 scope.go:117] "RemoveContainer" containerID="8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"
Sep 10 17:44:07.975584 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975558    2549 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-config\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.975584 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975584    2549 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5960e955-5d77-448d-9f32-b0c25a5e71de-config-out\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.975783 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975625    2549 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-web-config\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.975783 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975640    2549 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.975783 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975655    2549 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-grpc-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.975783 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975670    2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.975783 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975683    2549 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5960e955-5d77-448d-9f32-b0c25a5e71de-tls-assets\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.975783 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975712    2549 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-configmap-metrics-client-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.975783 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975727    2549 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.975783 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975741    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjc8v\" (UniqueName: \"kubernetes.io/projected/5960e955-5d77-448d-9f32-b0c25a5e71de-kube-api-access-fjc8v\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.975783 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975754    2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-db\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.975783 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975767    2549 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-metrics-client-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.975783 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975780    2549 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-kube-rbac-proxy\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.976419 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975795    2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.976419 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975810    2549 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.976419 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975824    2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.976419 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975837    2549 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5960e955-5d77-448d-9f32-b0c25a5e71de-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.976419 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.975852    2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5960e955-5d77-448d-9f32-b0c25a5e71de-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:44:07.977985 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.977969    2549 scope.go:117] "RemoveContainer" containerID="fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3"
Sep 10 17:44:07.985174 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985151    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:44:07.985436 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985423    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="kube-rbac-proxy"
Sep 10 17:44:07.985478 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985438    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="kube-rbac-proxy"
Sep 10 17:44:07.985478 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985450    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="init-config-reloader"
Sep 10 17:44:07.985478 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985455    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="init-config-reloader"
Sep 10 17:44:07.985478 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985461    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="config-reloader"
Sep 10 17:44:07.985478 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985469    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="config-reloader"
Sep 10 17:44:07.985615 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985484    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="kube-rbac-proxy-web"
Sep 10 17:44:07.985615 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985490    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="kube-rbac-proxy-web"
Sep 10 17:44:07.985615 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985499    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="prometheus"
Sep 10 17:44:07.985615 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985505    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="prometheus"
Sep 10 17:44:07.985615 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985512    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="kube-rbac-proxy-thanos"
Sep 10 17:44:07.985615 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985517    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="kube-rbac-proxy-thanos"
Sep 10 17:44:07.985615 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985522    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="thanos-sidecar"
Sep 10 17:44:07.985615 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985527    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="thanos-sidecar"
Sep 10 17:44:07.985615 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985573    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="thanos-sidecar"
Sep 10 17:44:07.985615 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985580    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="kube-rbac-proxy"
Sep 10 17:44:07.985615 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985586    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="config-reloader"
Sep 10 17:44:07.985615 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985591    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="kube-rbac-proxy-web"
Sep 10 17:44:07.985615 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985598    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="kube-rbac-proxy-thanos"
Sep 10 17:44:07.985615 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.985603    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" containerName="prometheus"
Sep 10 17:44:07.986363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.986341    2549 scope.go:117] "RemoveContainer" containerID="0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"
Sep 10 17:44:07.986793 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:44:07.986767    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6\": container with ID starting with 0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6 not found: ID does not exist" containerID="0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"
Sep 10 17:44:07.986873 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.986801    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"} err="failed to get container status \"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6\": rpc error: code = NotFound desc = could not find container \"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6\": container with ID starting with 0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6 not found: ID does not exist"
Sep 10 17:44:07.986873 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.986835    2549 scope.go:117] "RemoveContainer" containerID="220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"
Sep 10 17:44:07.987218 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:44:07.987192    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb\": container with ID starting with 220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb not found: ID does not exist" containerID="220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"
Sep 10 17:44:07.987353 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.987226    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"} err="failed to get container status \"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb\": rpc error: code = NotFound desc = could not find container \"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb\": container with ID starting with 220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb not found: ID does not exist"
Sep 10 17:44:07.987353 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.987248    2549 scope.go:117] "RemoveContainer" containerID="28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"
Sep 10 17:44:07.987659 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:44:07.987637    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe\": container with ID starting with 28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe not found: ID does not exist" containerID="28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"
Sep 10 17:44:07.987892 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.987666    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"} err="failed to get container status \"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe\": rpc error: code = NotFound desc = could not find container \"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe\": container with ID starting with 28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe not found: ID does not exist"
Sep 10 17:44:07.987892 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.987686    2549 scope.go:117] "RemoveContainer" containerID="86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"
Sep 10 17:44:07.987999 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:44:07.987970    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc\": container with ID starting with 86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc not found: ID does not exist" containerID="86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"
Sep 10 17:44:07.988049 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.988035    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"} err="failed to get container status \"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc\": rpc error: code = NotFound desc = could not find container \"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc\": container with ID starting with 86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc not found: ID does not exist"
Sep 10 17:44:07.988100 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.988054    2549 scope.go:117] "RemoveContainer" containerID="07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"
Sep 10 17:44:07.989903 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:44:07.989883    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62\": container with ID starting with 07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62 not found: ID does not exist" containerID="07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"
Sep 10 17:44:07.989989 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.989925    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"} err="failed to get container status \"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62\": rpc error: code = NotFound desc = could not find container \"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62\": container with ID starting with 07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62 not found: ID does not exist"
Sep 10 17:44:07.989989 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.989944    2549 scope.go:117] "RemoveContainer" containerID="8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"
Sep 10 17:44:07.990399 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:44:07.990354    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545\": container with ID starting with 8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545 not found: ID does not exist" containerID="8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"
Sep 10 17:44:07.990472 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.990414    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"} err="failed to get container status \"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545\": rpc error: code = NotFound desc = could not find container \"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545\": container with ID starting with 8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545 not found: ID does not exist"
Sep 10 17:44:07.990472 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.990434    2549 scope.go:117] "RemoveContainer" containerID="fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3"
Sep 10 17:44:07.990751 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:44:07.990730    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3\": container with ID starting with fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3 not found: ID does not exist" containerID="fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3"
Sep 10 17:44:07.990840 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.990754    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3"} err="failed to get container status \"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3\": rpc error: code = NotFound desc = could not find container \"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3\": container with ID starting with fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3 not found: ID does not exist"
Sep 10 17:44:07.990840 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.990768    2549 scope.go:117] "RemoveContainer" containerID="0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"
Sep 10 17:44:07.991111 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.991091    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"} err="failed to get container status \"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6\": rpc error: code = NotFound desc = could not find container \"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6\": container with ID starting with 0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6 not found: ID does not exist"
Sep 10 17:44:07.991111 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.991111    2549 scope.go:117] "RemoveContainer" containerID="220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"
Sep 10 17:44:07.991361 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.991344    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"} err="failed to get container status \"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb\": rpc error: code = NotFound desc = could not find container \"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb\": container with ID starting with 220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb not found: ID does not exist"
Sep 10 17:44:07.991361 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.991358    2549 scope.go:117] "RemoveContainer" containerID="28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"
Sep 10 17:44:07.991606 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.991588    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"} err="failed to get container status \"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe\": rpc error: code = NotFound desc = could not find container \"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe\": container with ID starting with 28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe not found: ID does not exist"
Sep 10 17:44:07.991650 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.991608    2549 scope.go:117] "RemoveContainer" containerID="86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"
Sep 10 17:44:07.991711 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.991682    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:07.991951 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.991864    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"} err="failed to get container status \"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc\": rpc error: code = NotFound desc = could not find container \"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc\": container with ID starting with 86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc not found: ID does not exist"
Sep 10 17:44:07.991951 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.991886    2549 scope.go:117] "RemoveContainer" containerID="07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"
Sep 10 17:44:07.993875 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.993684    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"} err="failed to get container status \"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62\": rpc error: code = NotFound desc = could not find container \"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62\": container with ID starting with 07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62 not found: ID does not exist"
Sep 10 17:44:07.994038 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.993877    2549 scope.go:117] "RemoveContainer" containerID="8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"
Sep 10 17:44:07.995849 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.994298    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"} err="failed to get container status \"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545\": rpc error: code = NotFound desc = could not find container \"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545\": container with ID starting with 8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545 not found: ID does not exist"
Sep 10 17:44:07.995849 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.994330    2549 scope.go:117] "RemoveContainer" containerID="fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3"
Sep 10 17:44:07.995849 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.995513    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3"} err="failed to get container status \"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3\": rpc error: code = NotFound desc = could not find container \"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3\": container with ID starting with fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3 not found: ID does not exist"
Sep 10 17:44:07.995849 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.995535    2549 scope.go:117] "RemoveContainer" containerID="0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"
Sep 10 17:44:07.995849 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.995565    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f6eegfg36d3q5\""
Sep 10 17:44:07.995849 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.995711    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 17:44:07.995849 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.995826    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 17:44:07.997106 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.996021    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 17:44:07.997106 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.996357    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"} err="failed to get container status \"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6\": rpc error: code = NotFound desc = could not find container \"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6\": container with ID starting with 0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6 not found: ID does not exist"
Sep 10 17:44:07.997106 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.996384    2549 scope.go:117] "RemoveContainer" containerID="220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"
Sep 10 17:44:07.997106 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.996373    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 17:44:07.997106 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.996500    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 17:44:07.997106 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.996628    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 17:44:07.997106 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.996969    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 17:44:07.997106 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.997057    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 17:44:07.997467 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.997299    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 17:44:07.997467 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.997412    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 17:44:07.997685 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.997665    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 17:44:07.997890 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.997871    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 17:44:07.998054 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.998026    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"} err="failed to get container status \"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb\": rpc error: code = NotFound desc = could not find container \"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb\": container with ID starting with 220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb not found: ID does not exist"
Sep 10 17:44:07.998234 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.998169    2549 scope.go:117] "RemoveContainer" containerID="28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"
Sep 10 17:44:07.998740 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.998569    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"} err="failed to get container status \"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe\": rpc error: code = NotFound desc = could not find container \"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe\": container with ID starting with 28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe not found: ID does not exist"
Sep 10 17:44:07.998740 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.998596    2549 scope.go:117] "RemoveContainer" containerID="86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"
Sep 10 17:44:07.998740 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.998716    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ltc8f\""
Sep 10 17:44:07.999635 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.999603    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"} err="failed to get container status \"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc\": rpc error: code = NotFound desc = could not find container \"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc\": container with ID starting with 86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc not found: ID does not exist"
Sep 10 17:44:07.999635 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:07.999634    2549 scope.go:117] "RemoveContainer" containerID="07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"
Sep 10 17:44:08.000036 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.000012    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"} err="failed to get container status \"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62\": rpc error: code = NotFound desc = could not find container \"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62\": container with ID starting with 07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62 not found: ID does not exist"
Sep 10 17:44:08.000116 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.000038    2549 scope.go:117] "RemoveContainer" containerID="8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"
Sep 10 17:44:08.000670 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.000644    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"} err="failed to get container status \"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545\": rpc error: code = NotFound desc = could not find container \"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545\": container with ID starting with 8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545 not found: ID does not exist"
Sep 10 17:44:08.000767 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.000672    2549 scope.go:117] "RemoveContainer" containerID="fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3"
Sep 10 17:44:08.000966 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.000946    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3"} err="failed to get container status \"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3\": rpc error: code = NotFound desc = could not find container \"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3\": container with ID starting with fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3 not found: ID does not exist"
Sep 10 17:44:08.001041 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.000968    2549 scope.go:117] "RemoveContainer" containerID="0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"
Sep 10 17:44:08.001226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.001206    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"} err="failed to get container status \"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6\": rpc error: code = NotFound desc = could not find container \"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6\": container with ID starting with 0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6 not found: ID does not exist"
Sep 10 17:44:08.001302 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.001228    2549 scope.go:117] "RemoveContainer" containerID="220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"
Sep 10 17:44:08.001427 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.001404    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 17:44:08.001516 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.001466    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"} err="failed to get container status \"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb\": rpc error: code = NotFound desc = could not find container \"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb\": container with ID starting with 220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb not found: ID does not exist"
Sep 10 17:44:08.001516 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.001491    2549 scope.go:117] "RemoveContainer" containerID="28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"
Sep 10 17:44:08.002018 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.001888    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"} err="failed to get container status \"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe\": rpc error: code = NotFound desc = could not find container \"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe\": container with ID starting with 28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe not found: ID does not exist"
Sep 10 17:44:08.002018 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.001914    2549 scope.go:117] "RemoveContainer" containerID="86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"
Sep 10 17:44:08.002480 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.002275    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"} err="failed to get container status \"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc\": rpc error: code = NotFound desc = could not find container \"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc\": container with ID starting with 86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc not found: ID does not exist"
Sep 10 17:44:08.002480 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.002304    2549 scope.go:117] "RemoveContainer" containerID="07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"
Sep 10 17:44:08.003534 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.002809    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"} err="failed to get container status \"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62\": rpc error: code = NotFound desc = could not find container \"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62\": container with ID starting with 07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62 not found: ID does not exist"
Sep 10 17:44:08.003534 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.002830    2549 scope.go:117] "RemoveContainer" containerID="8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"
Sep 10 17:44:08.003534 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.003094    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"} err="failed to get container status \"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545\": rpc error: code = NotFound desc = could not find container \"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545\": container with ID starting with 8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545 not found: ID does not exist"
Sep 10 17:44:08.003534 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.003115    2549 scope.go:117] "RemoveContainer" containerID="fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3"
Sep 10 17:44:08.003534 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.003467    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3"} err="failed to get container status \"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3\": rpc error: code = NotFound desc = could not find container \"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3\": container with ID starting with fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3 not found: ID does not exist"
Sep 10 17:44:08.003534 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.003489    2549 scope.go:117] "RemoveContainer" containerID="0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"
Sep 10 17:44:08.004636 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.004610    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"} err="failed to get container status \"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6\": rpc error: code = NotFound desc = could not find container \"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6\": container with ID starting with 0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6 not found: ID does not exist"
Sep 10 17:44:08.004636 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.004634    2549 scope.go:117] "RemoveContainer" containerID="220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"
Sep 10 17:44:08.004813 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.004714    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 17:44:08.004964 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.004930    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"} err="failed to get container status \"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb\": rpc error: code = NotFound desc = could not find container \"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb\": container with ID starting with 220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb not found: ID does not exist"
Sep 10 17:44:08.005069 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.004980    2549 scope.go:117] "RemoveContainer" containerID="28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"
Sep 10 17:44:08.005292 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.005273    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"} err="failed to get container status \"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe\": rpc error: code = NotFound desc = could not find container \"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe\": container with ID starting with 28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe not found: ID does not exist"
Sep 10 17:44:08.005377 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.005293    2549 scope.go:117] "RemoveContainer" containerID="86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"
Sep 10 17:44:08.005377 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.005324    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:44:08.005594 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.005570    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"} err="failed to get container status \"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc\": rpc error: code = NotFound desc = could not find container \"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc\": container with ID starting with 86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc not found: ID does not exist"
Sep 10 17:44:08.005672 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.005595    2549 scope.go:117] "RemoveContainer" containerID="07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"
Sep 10 17:44:08.005882 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.005864    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"} err="failed to get container status \"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62\": rpc error: code = NotFound desc = could not find container \"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62\": container with ID starting with 07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62 not found: ID does not exist"
Sep 10 17:44:08.005936 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.005884    2549 scope.go:117] "RemoveContainer" containerID="8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"
Sep 10 17:44:08.006196 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.006174    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"} err="failed to get container status \"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545\": rpc error: code = NotFound desc = could not find container \"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545\": container with ID starting with 8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545 not found: ID does not exist"
Sep 10 17:44:08.006267 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.006196    2549 scope.go:117] "RemoveContainer" containerID="fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3"
Sep 10 17:44:08.006478 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.006461    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3"} err="failed to get container status \"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3\": rpc error: code = NotFound desc = could not find container \"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3\": container with ID starting with fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3 not found: ID does not exist"
Sep 10 17:44:08.006521 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.006479    2549 scope.go:117] "RemoveContainer" containerID="0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"
Sep 10 17:44:08.006769 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.006746    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6"} err="failed to get container status \"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6\": rpc error: code = NotFound desc = could not find container \"0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6\": container with ID starting with 0ffa530716815ddbf1d832c4c95e386aca3cce9a270a5adb118795837995d5b6 not found: ID does not exist"
Sep 10 17:44:08.006847 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.006772    2549 scope.go:117] "RemoveContainer" containerID="220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"
Sep 10 17:44:08.007069 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.007049    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb"} err="failed to get container status \"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb\": rpc error: code = NotFound desc = could not find container \"220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb\": container with ID starting with 220b43c4fbc1867da0373e363abea293ddbb5b25235c3470236d0108f6d3daeb not found: ID does not exist"
Sep 10 17:44:08.007142 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.007070    2549 scope.go:117] "RemoveContainer" containerID="28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"
Sep 10 17:44:08.007370 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.007347    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe"} err="failed to get container status \"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe\": rpc error: code = NotFound desc = could not find container \"28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe\": container with ID starting with 28420f10d925050249991bf7ac7a970f453bd5f915b1a727db3ad81d72e78fbe not found: ID does not exist"
Sep 10 17:44:08.007424 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.007371    2549 scope.go:117] "RemoveContainer" containerID="86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"
Sep 10 17:44:08.007637 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.007620    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc"} err="failed to get container status \"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc\": rpc error: code = NotFound desc = could not find container \"86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc\": container with ID starting with 86f6d00bf91db90816662cbde1377424a4bbb2f6125f10825db49a6769ceaedc not found: ID does not exist"
Sep 10 17:44:08.007703 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.007637    2549 scope.go:117] "RemoveContainer" containerID="07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"
Sep 10 17:44:08.007885 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.007869    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62"} err="failed to get container status \"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62\": rpc error: code = NotFound desc = could not find container \"07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62\": container with ID starting with 07241ee817fcbbc0cb27bb05e355ce4af8cd06e54b710bf930b8c4684d42ef62 not found: ID does not exist"
Sep 10 17:44:08.007923 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.007884    2549 scope.go:117] "RemoveContainer" containerID="8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"
Sep 10 17:44:08.008091 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.008075    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545"} err="failed to get container status \"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545\": rpc error: code = NotFound desc = could not find container \"8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545\": container with ID starting with 8804faf5ee1595f45823492495f7526b0e0387835a19b3cff77de27ee28e0545 not found: ID does not exist"
Sep 10 17:44:08.008156 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.008091    2549 scope.go:117] "RemoveContainer" containerID="fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3"
Sep 10 17:44:08.008308 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.008293    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3"} err="failed to get container status \"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3\": rpc error: code = NotFound desc = could not find container \"fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3\": container with ID starting with fb61f37bdfdc2a55d7c651caf0a15b04c966feaab56ff17159c95ee3d5750ab3 not found: ID does not exist"
Sep 10 17:44:08.029761 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.029729    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5960e955-5d77-448d-9f32-b0c25a5e71de" path="/var/lib/kubelet/pods/5960e955-5d77-448d-9f32-b0c25a5e71de/volumes"
Sep 10 17:44:08.076761 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.076714    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.076761 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.076767    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.076761 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.076784    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077055 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.076881    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077055 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.076916    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077055 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.076940    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077055 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.076974    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077055 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.076997    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2bcb7a83-181a-4c86-8bb4-313deb152061-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077258 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.077074    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077258 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.077106    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077258 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.077131    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2bcb7a83-181a-4c86-8bb4-313deb152061-config-out\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077258 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.077154    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-web-config\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077258 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.077186    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077258 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.077235    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ppht\" (UniqueName: \"kubernetes.io/projected/2bcb7a83-181a-4c86-8bb4-313deb152061-kube-api-access-6ppht\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077509 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.077262    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077509 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.077279    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077509 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.077296    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077509 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.077344    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-config\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077509 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.077373    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2bcb7a83-181a-4c86-8bb4-313deb152061-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.077509 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.077398    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.178628 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178544    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.178628 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178585    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.178628 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178606    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.178875 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178632    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.178875 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178657    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2bcb7a83-181a-4c86-8bb4-313deb152061-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.178875 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178731    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.178875 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178762    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.178875 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178788    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2bcb7a83-181a-4c86-8bb4-313deb152061-config-out\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.178875 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178808    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-web-config\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.178875 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178829    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.178875 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178854    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ppht\" (UniqueName: \"kubernetes.io/projected/2bcb7a83-181a-4c86-8bb4-313deb152061-kube-api-access-6ppht\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.179251 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178884    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.179251 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178912    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.179251 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178938    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.179251 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178966    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-config\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.179251 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.178985    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2bcb7a83-181a-4c86-8bb4-313deb152061-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.179251 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.179012    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.179251 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.179058    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.179251 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.179087    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.179251 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.179108    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.179251 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.179190    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2bcb7a83-181a-4c86-8bb4-313deb152061-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.179749 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.179384    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.179749 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.179470    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.179749 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.179564    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.179909 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.179862    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.180243 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.180213    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.180366 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.180331    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.181941 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.181909    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2bcb7a83-181a-4c86-8bb4-313deb152061-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.182049 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.181955    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.182049 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.182014    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.182621 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.182293    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.182621 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.182529    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.182866 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.182818    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.183299 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.183265    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.183544 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.183527    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2bcb7a83-181a-4c86-8bb4-313deb152061-config-out\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.184461 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.184440    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-config\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.184583 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.184566    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.184733 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.184713    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2bcb7a83-181a-4c86-8bb4-313deb152061-web-config\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.185186 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.185169    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2bcb7a83-181a-4c86-8bb4-313deb152061-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.189908 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.189891    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ppht\" (UniqueName: \"kubernetes.io/projected/2bcb7a83-181a-4c86-8bb4-313deb152061-kube-api-access-6ppht\") pod \"prometheus-k8s-0\" (UID: \"2bcb7a83-181a-4c86-8bb4-313deb152061\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.307795 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.307755    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:44:08.590745 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.590724    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:44:08.593102 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:44:08.593076    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bcb7a83_181a_4c86_8bb4_313deb152061.slice/crio-3ce69a3e402e46fc3e4a0a4fa07c84a34d592564d613f67e616079df4953647e WatchSource:0}: Error finding container 3ce69a3e402e46fc3e4a0a4fa07c84a34d592564d613f67e616079df4953647e: Status 404 returned error can't find the container with id 3ce69a3e402e46fc3e4a0a4fa07c84a34d592564d613f67e616079df4953647e
Sep 10 17:44:08.928164 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.928084    2549 generic.go:356] "Generic (PLEG): container finished" podID="2bcb7a83-181a-4c86-8bb4-313deb152061" containerID="f1a3a29bbde0721f857634986335296e9c314cc47187e604294d84d250e7aa4f" exitCode=0
Sep 10 17:44:08.928164 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.928134    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bcb7a83-181a-4c86-8bb4-313deb152061","Type":"ContainerDied","Data":"f1a3a29bbde0721f857634986335296e9c314cc47187e604294d84d250e7aa4f"}
Sep 10 17:44:08.928543 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:08.928179    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bcb7a83-181a-4c86-8bb4-313deb152061","Type":"ContainerStarted","Data":"3ce69a3e402e46fc3e4a0a4fa07c84a34d592564d613f67e616079df4953647e"}
Sep 10 17:44:09.935555 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:09.935519    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bcb7a83-181a-4c86-8bb4-313deb152061","Type":"ContainerStarted","Data":"2a14ff36e6d79a3562317e2653f7363b2590b48b6d7982d0b58fc6c6d4b071d6"}
Sep 10 17:44:09.936039 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:09.935627    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bcb7a83-181a-4c86-8bb4-313deb152061","Type":"ContainerStarted","Data":"8b45a6828b1d2b2d4d6a1af1a88f9a74a4aff304a51f9d9f1626366bbf06fd13"}
Sep 10 17:44:09.936039 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:09.935677    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bcb7a83-181a-4c86-8bb4-313deb152061","Type":"ContainerStarted","Data":"cfe039cee50a1fe5589563d8d2889a27ad6714a83882f1dcbb47c8eb7428ae5e"}
Sep 10 17:44:09.936039 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:09.935740    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bcb7a83-181a-4c86-8bb4-313deb152061","Type":"ContainerStarted","Data":"7289aa1dac00aa97f7c7d93ad86e2177cd8eb781a623a691dac4a1a644f86d48"}
Sep 10 17:44:09.936039 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:09.935786    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bcb7a83-181a-4c86-8bb4-313deb152061","Type":"ContainerStarted","Data":"31e1c3b48ed3db443f163b098f6a89596b4a4af58a547015182e67bae192ce87"}
Sep 10 17:44:09.936039 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:09.935826    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bcb7a83-181a-4c86-8bb4-313deb152061","Type":"ContainerStarted","Data":"00a4bb00811bf78d7e9fcaa03e1e8d21bcef2946f7dd8dcf5d75722ef2ab3cca"}
Sep 10 17:44:09.965705 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:09.965637    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.944861131 podStartE2EDuration="2.965620918s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.759787ms" isStatefulPod=false podCreationTimestamp="2026-09-10 17:44:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:44:09.963531639 +0000 UTC m=+174.454583448" watchObservedRunningTime="2026-09-10 17:44:09.965620918 +0000 UTC m=+174.456672747"
Sep 10 17:44:13.307929 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:44:13.307891    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:45:08.308095 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:45:08.308060    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:45:08.323875 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:45:08.323848    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:45:09.177956 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:45:09.177930    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:46:15.912227 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:15.912201    2549 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 17:46:22.260795 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.260761    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"]
Sep 10 17:46:22.262854 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.262840    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:22.266998 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.266977    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 17:46:22.267125 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.267010    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-t5xjq\""
Sep 10 17:46:22.267125 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.267103    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 17:46:22.267241 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.267135    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 17:46:22.267297 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.267260    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 17:46:22.267350 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.267319    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 17:46:22.277319 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.277295    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"]
Sep 10 17:46:22.374156 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.374102    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/086300ad-3bb7-4aaa-b982-ab577f740ba8-manager-config\") pod \"lws-controller-manager-746f98bbcc-c276f\" (UID: \"086300ad-3bb7-4aaa-b982-ab577f740ba8\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:22.374156 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.374180    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/086300ad-3bb7-4aaa-b982-ab577f740ba8-cert\") pod \"lws-controller-manager-746f98bbcc-c276f\" (UID: \"086300ad-3bb7-4aaa-b982-ab577f740ba8\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:22.374404 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.374211    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/086300ad-3bb7-4aaa-b982-ab577f740ba8-metrics-cert\") pod \"lws-controller-manager-746f98bbcc-c276f\" (UID: \"086300ad-3bb7-4aaa-b982-ab577f740ba8\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:22.374404 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.374244    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcbl8\" (UniqueName: \"kubernetes.io/projected/086300ad-3bb7-4aaa-b982-ab577f740ba8-kube-api-access-jcbl8\") pod \"lws-controller-manager-746f98bbcc-c276f\" (UID: \"086300ad-3bb7-4aaa-b982-ab577f740ba8\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:22.475098 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.475042    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/086300ad-3bb7-4aaa-b982-ab577f740ba8-cert\") pod \"lws-controller-manager-746f98bbcc-c276f\" (UID: \"086300ad-3bb7-4aaa-b982-ab577f740ba8\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:22.475098 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.475100    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/086300ad-3bb7-4aaa-b982-ab577f740ba8-metrics-cert\") pod \"lws-controller-manager-746f98bbcc-c276f\" (UID: \"086300ad-3bb7-4aaa-b982-ab577f740ba8\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:22.475373 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.475220    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcbl8\" (UniqueName: \"kubernetes.io/projected/086300ad-3bb7-4aaa-b982-ab577f740ba8-kube-api-access-jcbl8\") pod \"lws-controller-manager-746f98bbcc-c276f\" (UID: \"086300ad-3bb7-4aaa-b982-ab577f740ba8\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:22.475373 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.475331    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/086300ad-3bb7-4aaa-b982-ab577f740ba8-manager-config\") pod \"lws-controller-manager-746f98bbcc-c276f\" (UID: \"086300ad-3bb7-4aaa-b982-ab577f740ba8\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:22.475947 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.475923    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/086300ad-3bb7-4aaa-b982-ab577f740ba8-manager-config\") pod \"lws-controller-manager-746f98bbcc-c276f\" (UID: \"086300ad-3bb7-4aaa-b982-ab577f740ba8\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:22.477562 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.477545    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/086300ad-3bb7-4aaa-b982-ab577f740ba8-metrics-cert\") pod \"lws-controller-manager-746f98bbcc-c276f\" (UID: \"086300ad-3bb7-4aaa-b982-ab577f740ba8\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:22.477562 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.477555    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/086300ad-3bb7-4aaa-b982-ab577f740ba8-cert\") pod \"lws-controller-manager-746f98bbcc-c276f\" (UID: \"086300ad-3bb7-4aaa-b982-ab577f740ba8\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:22.485317 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.485290    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcbl8\" (UniqueName: \"kubernetes.io/projected/086300ad-3bb7-4aaa-b982-ab577f740ba8-kube-api-access-jcbl8\") pod \"lws-controller-manager-746f98bbcc-c276f\" (UID: \"086300ad-3bb7-4aaa-b982-ab577f740ba8\") " pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:22.572065 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.571977    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:22.843151 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.843073    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"]
Sep 10 17:46:22.846064 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:46:22.846039    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod086300ad_3bb7_4aaa_b982_ab577f740ba8.slice/crio-d7eb5a1bdc46f2dfe8b687bf8b37f48ef5fbcbb907677374f2b82391767f41e1 WatchSource:0}: Error finding container d7eb5a1bdc46f2dfe8b687bf8b37f48ef5fbcbb907677374f2b82391767f41e1: Status 404 returned error can't find the container with id d7eb5a1bdc46f2dfe8b687bf8b37f48ef5fbcbb907677374f2b82391767f41e1
Sep 10 17:46:22.847836 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:22.847819    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:46:23.438083 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:23.438051    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f" event={"ID":"086300ad-3bb7-4aaa-b982-ab577f740ba8","Type":"ContainerStarted","Data":"d7eb5a1bdc46f2dfe8b687bf8b37f48ef5fbcbb907677374f2b82391767f41e1"}
Sep 10 17:46:25.448420 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.448380    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f" event={"ID":"086300ad-3bb7-4aaa-b982-ab577f740ba8","Type":"ContainerStarted","Data":"3f6aa4118b03e5df05cae92c4f21bdfe3dfbf5ad09523f7dc854698c48c4071b"}
Sep 10 17:46:25.448903 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.448508    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:25.474438 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.474386    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f" podStartSLOduration=1.04396329 podStartE2EDuration="3.474370452s" totalImagesPullingTime="2.430407162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:46:25.47221066 +0000 UTC m=+309.963262467" watchObservedRunningTime="2026-09-10 17:46:25.474370452 +0000 UTC m=+309.965422259"
Sep 10 17:46:25.839476 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.839441    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"]
Sep 10 17:46:25.841857 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.841840    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:25.845738 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.845718    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 17:46:25.845858 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.845718    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 17:46:25.845858 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.845766    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kbvxn\""
Sep 10 17:46:25.846150 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.846136    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 17:46:25.848131 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.848112    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 17:46:25.861785 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.861759    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"]
Sep 10 17:46:25.908226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.908185    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7f310d90-52be-4156-8478-084ecd2f768d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-556447cf57-n8n46\" (UID: \"7f310d90-52be-4156-8478-084ecd2f768d\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:25.908226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.908225    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7f310d90-52be-4156-8478-084ecd2f768d-manifests\") pod \"opendatahub-operator-controller-manager-556447cf57-n8n46\" (UID: \"7f310d90-52be-4156-8478-084ecd2f768d\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:25.908452 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.908246    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7f310d90-52be-4156-8478-084ecd2f768d-webhook-cert\") pod \"opendatahub-operator-controller-manager-556447cf57-n8n46\" (UID: \"7f310d90-52be-4156-8478-084ecd2f768d\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:25.908452 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.908302    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c842m\" (UniqueName: \"kubernetes.io/projected/7f310d90-52be-4156-8478-084ecd2f768d-kube-api-access-c842m\") pod \"opendatahub-operator-controller-manager-556447cf57-n8n46\" (UID: \"7f310d90-52be-4156-8478-084ecd2f768d\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:25.908452 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:25.908374    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f310d90-52be-4156-8478-084ecd2f768d-tmp\") pod \"opendatahub-operator-controller-manager-556447cf57-n8n46\" (UID: \"7f310d90-52be-4156-8478-084ecd2f768d\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:26.009510 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:26.009468    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f310d90-52be-4156-8478-084ecd2f768d-tmp\") pod \"opendatahub-operator-controller-manager-556447cf57-n8n46\" (UID: \"7f310d90-52be-4156-8478-084ecd2f768d\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:26.009599 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:26.009569    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7f310d90-52be-4156-8478-084ecd2f768d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-556447cf57-n8n46\" (UID: \"7f310d90-52be-4156-8478-084ecd2f768d\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:26.009650 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:26.009598    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7f310d90-52be-4156-8478-084ecd2f768d-manifests\") pod \"opendatahub-operator-controller-manager-556447cf57-n8n46\" (UID: \"7f310d90-52be-4156-8478-084ecd2f768d\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:26.009650 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:26.009620    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7f310d90-52be-4156-8478-084ecd2f768d-webhook-cert\") pod \"opendatahub-operator-controller-manager-556447cf57-n8n46\" (UID: \"7f310d90-52be-4156-8478-084ecd2f768d\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:26.009782 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:26.009672    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c842m\" (UniqueName: \"kubernetes.io/projected/7f310d90-52be-4156-8478-084ecd2f768d-kube-api-access-c842m\") pod \"opendatahub-operator-controller-manager-556447cf57-n8n46\" (UID: \"7f310d90-52be-4156-8478-084ecd2f768d\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:26.010004 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:26.009974    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7f310d90-52be-4156-8478-084ecd2f768d-manifests\") pod \"opendatahub-operator-controller-manager-556447cf57-n8n46\" (UID: \"7f310d90-52be-4156-8478-084ecd2f768d\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:26.011913 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:26.011891    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f310d90-52be-4156-8478-084ecd2f768d-tmp\") pod \"opendatahub-operator-controller-manager-556447cf57-n8n46\" (UID: \"7f310d90-52be-4156-8478-084ecd2f768d\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:26.012138 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:26.012118    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7f310d90-52be-4156-8478-084ecd2f768d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-556447cf57-n8n46\" (UID: \"7f310d90-52be-4156-8478-084ecd2f768d\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:26.012201 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:26.012148    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7f310d90-52be-4156-8478-084ecd2f768d-webhook-cert\") pod \"opendatahub-operator-controller-manager-556447cf57-n8n46\" (UID: \"7f310d90-52be-4156-8478-084ecd2f768d\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:26.027030 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:26.027009    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c842m\" (UniqueName: \"kubernetes.io/projected/7f310d90-52be-4156-8478-084ecd2f768d-kube-api-access-c842m\") pod \"opendatahub-operator-controller-manager-556447cf57-n8n46\" (UID: \"7f310d90-52be-4156-8478-084ecd2f768d\") " pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:26.151499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:26.151404    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:26.435499 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:26.435411    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"]
Sep 10 17:46:26.439223 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:46:26.439195    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f310d90_52be_4156_8478_084ecd2f768d.slice/crio-93b6189c79b1278a9ae5e7eec78659041af0f9c479a4b02b1578b0480264a6a3 WatchSource:0}: Error finding container 93b6189c79b1278a9ae5e7eec78659041af0f9c479a4b02b1578b0480264a6a3: Status 404 returned error can't find the container with id 93b6189c79b1278a9ae5e7eec78659041af0f9c479a4b02b1578b0480264a6a3
Sep 10 17:46:26.454167 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:26.454126    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46" event={"ID":"7f310d90-52be-4156-8478-084ecd2f768d","Type":"ContainerStarted","Data":"93b6189c79b1278a9ae5e7eec78659041af0f9c479a4b02b1578b0480264a6a3"}
Sep 10 17:46:30.473020 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:30.472972    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46" event={"ID":"7f310d90-52be-4156-8478-084ecd2f768d","Type":"ContainerStarted","Data":"f5b197dec3bd98cdea2fdd5975c18f63bd5a0d94723884262bf3b32cdd4b287a"}
Sep 10 17:46:31.480453 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:31.480419    2549 generic.go:356] "Generic (PLEG): container finished" podID="7f310d90-52be-4156-8478-084ecd2f768d" containerID="f5b197dec3bd98cdea2fdd5975c18f63bd5a0d94723884262bf3b32cdd4b287a" exitCode=0
Sep 10 17:46:31.480867 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:31.480504    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46" event={"ID":"7f310d90-52be-4156-8478-084ecd2f768d","Type":"ContainerDied","Data":"f5b197dec3bd98cdea2fdd5975c18f63bd5a0d94723884262bf3b32cdd4b287a"}
Sep 10 17:46:32.486837 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:32.486802    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46" event={"ID":"7f310d90-52be-4156-8478-084ecd2f768d","Type":"ContainerStarted","Data":"d206be6aa55f5b8307e2b9f84e083cd207df8be3e223a906d52c95efea7964e1"}
Sep 10 17:46:32.487226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:32.487050    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:32.510399 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:32.510339    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46" podStartSLOduration=3.139116835 podStartE2EDuration="7.510322165s" totalImagesPullingTime="3.319172084s" totalInitContainerRuntime="1.052033246s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:46:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:46:32.508382726 +0000 UTC m=+316.999434534" watchObservedRunningTime="2026-09-10 17:46:32.510322165 +0000 UTC m=+317.001374033"
Sep 10 17:46:36.456166 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:36.456134    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-746f98bbcc-c276f"
Sep 10 17:46:43.493805 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:43.493769    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-556447cf57-n8n46"
Sep 10 17:46:45.862815 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:45.862782    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q"]
Sep 10 17:46:45.870058 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:45.870029    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q"
Sep 10 17:46:45.873486 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:45.873461    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 17:46:45.874711 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:45.874670    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wnt8s\""
Sep 10 17:46:45.874828 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:45.874670    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 17:46:45.874828 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:45.874727    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 17:46:45.874828 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:45.874804    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 17:46:45.879911 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:45.879889    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q"]
Sep 10 17:46:45.983367 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:45.983333    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b982c773-c239-4c45-be2c-6614850d2215-tmp\") pod \"kube-auth-proxy-5c775fbbf-z4v6q\" (UID: \"b982c773-c239-4c45-be2c-6614850d2215\") " pod="openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q"
Sep 10 17:46:45.983625 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:45.983383    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqg87\" (UniqueName: \"kubernetes.io/projected/b982c773-c239-4c45-be2c-6614850d2215-kube-api-access-pqg87\") pod \"kube-auth-proxy-5c775fbbf-z4v6q\" (UID: \"b982c773-c239-4c45-be2c-6614850d2215\") " pod="openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q"
Sep 10 17:46:45.983625 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:45.983402    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b982c773-c239-4c45-be2c-6614850d2215-tls-certs\") pod \"kube-auth-proxy-5c775fbbf-z4v6q\" (UID: \"b982c773-c239-4c45-be2c-6614850d2215\") " pod="openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q"
Sep 10 17:46:46.084177 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:46.084122    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b982c773-c239-4c45-be2c-6614850d2215-tmp\") pod \"kube-auth-proxy-5c775fbbf-z4v6q\" (UID: \"b982c773-c239-4c45-be2c-6614850d2215\") " pod="openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q"
Sep 10 17:46:46.084177 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:46.084187    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqg87\" (UniqueName: \"kubernetes.io/projected/b982c773-c239-4c45-be2c-6614850d2215-kube-api-access-pqg87\") pod \"kube-auth-proxy-5c775fbbf-z4v6q\" (UID: \"b982c773-c239-4c45-be2c-6614850d2215\") " pod="openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q"
Sep 10 17:46:46.084426 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:46.084207    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b982c773-c239-4c45-be2c-6614850d2215-tls-certs\") pod \"kube-auth-proxy-5c775fbbf-z4v6q\" (UID: \"b982c773-c239-4c45-be2c-6614850d2215\") " pod="openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q"
Sep 10 17:46:46.086406 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:46.086375    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b982c773-c239-4c45-be2c-6614850d2215-tmp\") pod \"kube-auth-proxy-5c775fbbf-z4v6q\" (UID: \"b982c773-c239-4c45-be2c-6614850d2215\") " pod="openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q"
Sep 10 17:46:46.086614 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:46.086593    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b982c773-c239-4c45-be2c-6614850d2215-tls-certs\") pod \"kube-auth-proxy-5c775fbbf-z4v6q\" (UID: \"b982c773-c239-4c45-be2c-6614850d2215\") " pod="openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q"
Sep 10 17:46:46.103942 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:46.103916    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqg87\" (UniqueName: \"kubernetes.io/projected/b982c773-c239-4c45-be2c-6614850d2215-kube-api-access-pqg87\") pod \"kube-auth-proxy-5c775fbbf-z4v6q\" (UID: \"b982c773-c239-4c45-be2c-6614850d2215\") " pod="openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q"
Sep 10 17:46:46.180808 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:46.180728    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q"
Sep 10 17:46:46.456144 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:46.456120    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q"]
Sep 10 17:46:46.458357 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:46:46.458327    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb982c773_c239_4c45_be2c_6614850d2215.slice/crio-72b21d1835277276f9e6ae5da58d9e13ace155a988e203ff2f1f2dab9e3005b6 WatchSource:0}: Error finding container 72b21d1835277276f9e6ae5da58d9e13ace155a988e203ff2f1f2dab9e3005b6: Status 404 returned error can't find the container with id 72b21d1835277276f9e6ae5da58d9e13ace155a988e203ff2f1f2dab9e3005b6
Sep 10 17:46:46.543523 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:46.543479    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q" event={"ID":"b982c773-c239-4c45-be2c-6614850d2215","Type":"ContainerStarted","Data":"72b21d1835277276f9e6ae5da58d9e13ace155a988e203ff2f1f2dab9e3005b6"}
Sep 10 17:46:49.559478 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:49.559438    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q" event={"ID":"b982c773-c239-4c45-be2c-6614850d2215","Type":"ContainerStarted","Data":"0f6c4a2de16c7f465b8a6db98ee36d96050c3076a1628c19689be6d832dd6065"}
Sep 10 17:46:49.577949 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:46:49.577894    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5c775fbbf-z4v6q" podStartSLOduration=1.68854926 podStartE2EDuration="4.577878689s" totalImagesPullingTime="2.889329429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:46:49.576788523 +0000 UTC m=+334.067840330" watchObservedRunningTime="2026-09-10 17:46:49.577878689 +0000 UTC m=+334.068930503"
Sep 10 17:48:39.721859 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:39.721782    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"]
Sep 10 17:48:39.725008 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:39.724990    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"
Sep 10 17:48:39.727560 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:39.727534    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gs9tl\""
Sep 10 17:48:39.727847 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:39.727830    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 17:48:39.727910 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:39.727892    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 17:48:39.738977 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:39.738948    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"]
Sep 10 17:48:39.757767 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:39.757736    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/578f1bb5-707f-4aa6-83e0-68f3d523d778-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-svw5c\" (UID: \"578f1bb5-707f-4aa6-83e0-68f3d523d778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"
Sep 10 17:48:39.757893 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:39.757828    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zts24\" (UniqueName: \"kubernetes.io/projected/578f1bb5-707f-4aa6-83e0-68f3d523d778-kube-api-access-zts24\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-svw5c\" (UID: \"578f1bb5-707f-4aa6-83e0-68f3d523d778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"
Sep 10 17:48:39.858449 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:39.858415    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zts24\" (UniqueName: \"kubernetes.io/projected/578f1bb5-707f-4aa6-83e0-68f3d523d778-kube-api-access-zts24\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-svw5c\" (UID: \"578f1bb5-707f-4aa6-83e0-68f3d523d778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"
Sep 10 17:48:39.858604 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:39.858462    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/578f1bb5-707f-4aa6-83e0-68f3d523d778-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-svw5c\" (UID: \"578f1bb5-707f-4aa6-83e0-68f3d523d778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"
Sep 10 17:48:39.858829 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:39.858811    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/578f1bb5-707f-4aa6-83e0-68f3d523d778-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-svw5c\" (UID: \"578f1bb5-707f-4aa6-83e0-68f3d523d778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"
Sep 10 17:48:39.866656 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:39.866633    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zts24\" (UniqueName: \"kubernetes.io/projected/578f1bb5-707f-4aa6-83e0-68f3d523d778-kube-api-access-zts24\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-svw5c\" (UID: \"578f1bb5-707f-4aa6-83e0-68f3d523d778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"
Sep 10 17:48:40.034429 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:40.034355    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"
Sep 10 17:48:40.310601 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:40.310576    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"]
Sep 10 17:48:40.311993 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:48:40.311968    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod578f1bb5_707f_4aa6_83e0_68f3d523d778.slice/crio-dd1ca22d9bebd94bb13b13709854a7719540c5692b9eafed1e7155349200db10 WatchSource:0}: Error finding container dd1ca22d9bebd94bb13b13709854a7719540c5692b9eafed1e7155349200db10: Status 404 returned error can't find the container with id dd1ca22d9bebd94bb13b13709854a7719540c5692b9eafed1e7155349200db10
Sep 10 17:48:40.999749 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:40.999712    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"]
Sep 10 17:48:41.005531 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:41.005507    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"]
Sep 10 17:48:41.025356 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:41.025326    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"]
Sep 10 17:48:41.029011 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:41.028987    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"
Sep 10 17:48:41.045976 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:41.045940    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"]
Sep 10 17:48:41.067310 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:41.067268    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb45e509-9f09-4ccd-8714-4c8bcace8ec0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-q5qth\" (UID: \"fb45e509-9f09-4ccd-8714-4c8bcace8ec0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"
Sep 10 17:48:41.067480 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:41.067337    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkjdm\" (UniqueName: \"kubernetes.io/projected/fb45e509-9f09-4ccd-8714-4c8bcace8ec0-kube-api-access-fkjdm\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-q5qth\" (UID: \"fb45e509-9f09-4ccd-8714-4c8bcace8ec0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"
Sep 10 17:48:41.167905 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:41.167866    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb45e509-9f09-4ccd-8714-4c8bcace8ec0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-q5qth\" (UID: \"fb45e509-9f09-4ccd-8714-4c8bcace8ec0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"
Sep 10 17:48:41.168094 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:41.167915    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkjdm\" (UniqueName: \"kubernetes.io/projected/fb45e509-9f09-4ccd-8714-4c8bcace8ec0-kube-api-access-fkjdm\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-q5qth\" (UID: \"fb45e509-9f09-4ccd-8714-4c8bcace8ec0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"
Sep 10 17:48:41.168316 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:41.168295    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb45e509-9f09-4ccd-8714-4c8bcace8ec0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-q5qth\" (UID: \"fb45e509-9f09-4ccd-8714-4c8bcace8ec0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"
Sep 10 17:48:41.178032 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:41.178005    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkjdm\" (UniqueName: \"kubernetes.io/projected/fb45e509-9f09-4ccd-8714-4c8bcace8ec0-kube-api-access-fkjdm\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-q5qth\" (UID: \"fb45e509-9f09-4ccd-8714-4c8bcace8ec0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"
Sep 10 17:48:41.342905 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:41.342817    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"
Sep 10 17:48:41.638875 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:41.638840    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"]
Sep 10 17:48:41.640547 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:48:41.640519    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb45e509_9f09_4ccd_8714_4c8bcace8ec0.slice/crio-5d974f891d517d2620ab6d8532115143fcfffa2ae453be7a8941ae3cd2d8e1df WatchSource:0}: Error finding container 5d974f891d517d2620ab6d8532115143fcfffa2ae453be7a8941ae3cd2d8e1df: Status 404 returned error can't find the container with id 5d974f891d517d2620ab6d8532115143fcfffa2ae453be7a8941ae3cd2d8e1df
Sep 10 17:48:42.040232 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:42.040189    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth" event={"ID":"fb45e509-9f09-4ccd-8714-4c8bcace8ec0","Type":"ContainerStarted","Data":"5d974f891d517d2620ab6d8532115143fcfffa2ae453be7a8941ae3cd2d8e1df"}
Sep 10 17:48:45.060956 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:45.060867    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth" event={"ID":"fb45e509-9f09-4ccd-8714-4c8bcace8ec0","Type":"ContainerStarted","Data":"a77100ea99357db4d3205901294dbc8885ce0cab09db86b98fdd9c621d665074"}
Sep 10 17:48:45.061377 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:45.060977    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"
Sep 10 17:48:45.062756 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:45.062730    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c" podUID="578f1bb5-707f-4aa6-83e0-68f3d523d778" containerName="manager" containerID="cri-o://bb49340e2a48ba719aa65bfa53f62a2356702e1547ca1bd2373ae6f936a02886" gracePeriod=2
Sep 10 17:48:45.082995 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:45.082946    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth" podStartSLOduration=1.007945584 podStartE2EDuration="4.082928011s" totalImagesPullingTime="3.074982427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:48:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:48:45.080286029 +0000 UTC m=+449.571337834" watchObservedRunningTime="2026-09-10 17:48:45.082928011 +0000 UTC m=+449.573979818"
Sep 10 17:48:45.274472 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:45.274443    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"
Sep 10 17:48:45.278269 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:48:45.278234    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-svw5c\" is forbidden: User \"system:node:ip-10-0-139-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-245.ec2.internal' and this object" podUID="578f1bb5-707f-4aa6-83e0-68f3d523d778" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"
Sep 10 17:48:45.301323 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:45.301293    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zts24\" (UniqueName: \"kubernetes.io/projected/578f1bb5-707f-4aa6-83e0-68f3d523d778-kube-api-access-zts24\") pod \"578f1bb5-707f-4aa6-83e0-68f3d523d778\" (UID: \"578f1bb5-707f-4aa6-83e0-68f3d523d778\") "
Sep 10 17:48:45.301447 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:45.301355    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/578f1bb5-707f-4aa6-83e0-68f3d523d778-extensions-socket-volume\") pod \"578f1bb5-707f-4aa6-83e0-68f3d523d778\" (UID: \"578f1bb5-707f-4aa6-83e0-68f3d523d778\") "
Sep 10 17:48:45.301618 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:45.301596    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/578f1bb5-707f-4aa6-83e0-68f3d523d778-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "578f1bb5-707f-4aa6-83e0-68f3d523d778" (UID: "578f1bb5-707f-4aa6-83e0-68f3d523d778"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:48:45.303393 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:45.303374    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/578f1bb5-707f-4aa6-83e0-68f3d523d778-kube-api-access-zts24" (OuterVolumeSpecName: "kube-api-access-zts24") pod "578f1bb5-707f-4aa6-83e0-68f3d523d778" (UID: "578f1bb5-707f-4aa6-83e0-68f3d523d778"). InnerVolumeSpecName "kube-api-access-zts24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:48:45.402847 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:45.402755    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zts24\" (UniqueName: \"kubernetes.io/projected/578f1bb5-707f-4aa6-83e0-68f3d523d778-kube-api-access-zts24\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:48:45.402847 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:45.402790    2549 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/578f1bb5-707f-4aa6-83e0-68f3d523d778-extensions-socket-volume\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:48:46.027280 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:48:46.027237    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-svw5c\" is forbidden: User \"system:node:ip-10-0-139-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-245.ec2.internal' and this object" podUID="578f1bb5-707f-4aa6-83e0-68f3d523d778" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"
Sep 10 17:48:46.028364 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:46.028342    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="578f1bb5-707f-4aa6-83e0-68f3d523d778" path="/var/lib/kubelet/pods/578f1bb5-707f-4aa6-83e0-68f3d523d778/volumes"
Sep 10 17:48:46.068290 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:46.068256    2549 generic.go:356] "Generic (PLEG): container finished" podID="578f1bb5-707f-4aa6-83e0-68f3d523d778" containerID="bb49340e2a48ba719aa65bfa53f62a2356702e1547ca1bd2373ae6f936a02886" exitCode=2
Sep 10 17:48:46.068793 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:46.068305    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"
Sep 10 17:48:46.068793 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:46.068356    2549 scope.go:117] "RemoveContainer" containerID="bb49340e2a48ba719aa65bfa53f62a2356702e1547ca1bd2373ae6f936a02886"
Sep 10 17:48:46.073265 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:48:46.073235    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-svw5c\" is forbidden: User \"system:node:ip-10-0-139-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-245.ec2.internal' and this object" podUID="578f1bb5-707f-4aa6-83e0-68f3d523d778" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-svw5c"
Sep 10 17:48:46.076574 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:46.076559    2549 scope.go:117] "RemoveContainer" containerID="bb49340e2a48ba719aa65bfa53f62a2356702e1547ca1bd2373ae6f936a02886"
Sep 10 17:48:46.076853 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:48:46.076834    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb49340e2a48ba719aa65bfa53f62a2356702e1547ca1bd2373ae6f936a02886\": container with ID starting with bb49340e2a48ba719aa65bfa53f62a2356702e1547ca1bd2373ae6f936a02886 not found: ID does not exist" containerID="bb49340e2a48ba719aa65bfa53f62a2356702e1547ca1bd2373ae6f936a02886"
Sep 10 17:48:46.076908 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:46.076862    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb49340e2a48ba719aa65bfa53f62a2356702e1547ca1bd2373ae6f936a02886"} err="failed to get container status \"bb49340e2a48ba719aa65bfa53f62a2356702e1547ca1bd2373ae6f936a02886\": rpc error: code = NotFound desc = could not find container \"bb49340e2a48ba719aa65bfa53f62a2356702e1547ca1bd2373ae6f936a02886\": container with ID starting with bb49340e2a48ba719aa65bfa53f62a2356702e1547ca1bd2373ae6f936a02886 not found: ID does not exist"
Sep 10 17:48:56.070832 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:48:56.070797    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"
Sep 10 17:49:09.026956 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:09.026927    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"]
Sep 10 17:49:09.027368 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:09.027119    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth" podUID="fb45e509-9f09-4ccd-8714-4c8bcace8ec0" containerName="manager" containerID="cri-o://a77100ea99357db4d3205901294dbc8885ce0cab09db86b98fdd9c621d665074" gracePeriod=10
Sep 10 17:49:09.178641 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:09.178605    2549 generic.go:356] "Generic (PLEG): container finished" podID="fb45e509-9f09-4ccd-8714-4c8bcace8ec0" containerID="a77100ea99357db4d3205901294dbc8885ce0cab09db86b98fdd9c621d665074" exitCode=0
Sep 10 17:49:09.178838 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:09.178687    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth" event={"ID":"fb45e509-9f09-4ccd-8714-4c8bcace8ec0","Type":"ContainerDied","Data":"a77100ea99357db4d3205901294dbc8885ce0cab09db86b98fdd9c621d665074"}
Sep 10 17:49:09.270454 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:09.270431    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"
Sep 10 17:49:09.425799 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:09.425714    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb45e509-9f09-4ccd-8714-4c8bcace8ec0-extensions-socket-volume\") pod \"fb45e509-9f09-4ccd-8714-4c8bcace8ec0\" (UID: \"fb45e509-9f09-4ccd-8714-4c8bcace8ec0\") "
Sep 10 17:49:09.425971 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:09.425809    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkjdm\" (UniqueName: \"kubernetes.io/projected/fb45e509-9f09-4ccd-8714-4c8bcace8ec0-kube-api-access-fkjdm\") pod \"fb45e509-9f09-4ccd-8714-4c8bcace8ec0\" (UID: \"fb45e509-9f09-4ccd-8714-4c8bcace8ec0\") "
Sep 10 17:49:09.426200 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:09.426174    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb45e509-9f09-4ccd-8714-4c8bcace8ec0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fb45e509-9f09-4ccd-8714-4c8bcace8ec0" (UID: "fb45e509-9f09-4ccd-8714-4c8bcace8ec0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:49:09.427946 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:09.427911    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb45e509-9f09-4ccd-8714-4c8bcace8ec0-kube-api-access-fkjdm" (OuterVolumeSpecName: "kube-api-access-fkjdm") pod "fb45e509-9f09-4ccd-8714-4c8bcace8ec0" (UID: "fb45e509-9f09-4ccd-8714-4c8bcace8ec0"). InnerVolumeSpecName "kube-api-access-fkjdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:49:09.526498 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:09.526456    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkjdm\" (UniqueName: \"kubernetes.io/projected/fb45e509-9f09-4ccd-8714-4c8bcace8ec0-kube-api-access-fkjdm\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:49:09.526498 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:09.526491    2549 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb45e509-9f09-4ccd-8714-4c8bcace8ec0-extensions-socket-volume\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:49:10.184397 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:10.184367    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"
Sep 10 17:49:10.184397 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:10.184389    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth" event={"ID":"fb45e509-9f09-4ccd-8714-4c8bcace8ec0","Type":"ContainerDied","Data":"5d974f891d517d2620ab6d8532115143fcfffa2ae453be7a8941ae3cd2d8e1df"}
Sep 10 17:49:10.184909 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:10.184443    2549 scope.go:117] "RemoveContainer" containerID="a77100ea99357db4d3205901294dbc8885ce0cab09db86b98fdd9c621d665074"
Sep 10 17:49:10.206730 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:10.206702    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"]
Sep 10 17:49:10.211245 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:10.211220    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-q5qth"]
Sep 10 17:49:12.029360 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:12.029325    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb45e509-9f09-4ccd-8714-4c8bcace8ec0" path="/var/lib/kubelet/pods/fb45e509-9f09-4ccd-8714-4c8bcace8ec0/volumes"
Sep 10 17:49:28.163221 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.163187    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:49:28.163591 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.163483    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb45e509-9f09-4ccd-8714-4c8bcace8ec0" containerName="manager"
Sep 10 17:49:28.163591 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.163495    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb45e509-9f09-4ccd-8714-4c8bcace8ec0" containerName="manager"
Sep 10 17:49:28.163591 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.163517    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="578f1bb5-707f-4aa6-83e0-68f3d523d778" containerName="manager"
Sep 10 17:49:28.163591 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.163523    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="578f1bb5-707f-4aa6-83e0-68f3d523d778" containerName="manager"
Sep 10 17:49:28.163591 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.163568    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="578f1bb5-707f-4aa6-83e0-68f3d523d778" containerName="manager"
Sep 10 17:49:28.163591 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.163577    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb45e509-9f09-4ccd-8714-4c8bcace8ec0" containerName="manager"
Sep 10 17:49:28.168307 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.168291    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-ld75g"
Sep 10 17:49:28.171052 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.171030    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-c9gdn\""
Sep 10 17:49:28.171173 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.171051    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 17:49:28.172155 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.172134    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 17:49:28.172255 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.172166    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 17:49:28.176510 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.176488    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:49:28.186341 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.186317    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzqq7\" (UniqueName: \"kubernetes.io/projected/12cca286-9d95-486f-9f8e-b0f4e530cb24-kube-api-access-pzqq7\") pod \"limitador-limitador-84994f7ffc-ld75g\" (UID: \"12cca286-9d95-486f-9f8e-b0f4e530cb24\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ld75g"
Sep 10 17:49:28.186450 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.186366    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/12cca286-9d95-486f-9f8e-b0f4e530cb24-config-file\") pod \"limitador-limitador-84994f7ffc-ld75g\" (UID: \"12cca286-9d95-486f-9f8e-b0f4e530cb24\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ld75g"
Sep 10 17:49:28.227636 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.227603    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:49:28.287560 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.287528    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzqq7\" (UniqueName: \"kubernetes.io/projected/12cca286-9d95-486f-9f8e-b0f4e530cb24-kube-api-access-pzqq7\") pod \"limitador-limitador-84994f7ffc-ld75g\" (UID: \"12cca286-9d95-486f-9f8e-b0f4e530cb24\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ld75g"
Sep 10 17:49:28.287798 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.287584    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/12cca286-9d95-486f-9f8e-b0f4e530cb24-config-file\") pod \"limitador-limitador-84994f7ffc-ld75g\" (UID: \"12cca286-9d95-486f-9f8e-b0f4e530cb24\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ld75g"
Sep 10 17:49:28.288198 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.288179    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/12cca286-9d95-486f-9f8e-b0f4e530cb24-config-file\") pod \"limitador-limitador-84994f7ffc-ld75g\" (UID: \"12cca286-9d95-486f-9f8e-b0f4e530cb24\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ld75g"
Sep 10 17:49:28.295607 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.295586    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzqq7\" (UniqueName: \"kubernetes.io/projected/12cca286-9d95-486f-9f8e-b0f4e530cb24-kube-api-access-pzqq7\") pod \"limitador-limitador-84994f7ffc-ld75g\" (UID: \"12cca286-9d95-486f-9f8e-b0f4e530cb24\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ld75g"
Sep 10 17:49:28.478726 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.478671    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-ld75g"
Sep 10 17:49:28.713855 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.713821    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-mgxqf"]
Sep 10 17:49:28.717458 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.717435    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mgxqf"
Sep 10 17:49:28.719942 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.719922    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8v6q4\""
Sep 10 17:49:28.723877 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.723856    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mgxqf"]
Sep 10 17:49:28.759414 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.759330    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:49:28.762522 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:49:28.762499    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12cca286_9d95_486f_9f8e_b0f4e530cb24.slice/crio-7b722a2c4547829e944d4093100c0a02b47c1fd32a04fd7b0bee6da7cee30843 WatchSource:0}: Error finding container 7b722a2c4547829e944d4093100c0a02b47c1fd32a04fd7b0bee6da7cee30843: Status 404 returned error can't find the container with id 7b722a2c4547829e944d4093100c0a02b47c1fd32a04fd7b0bee6da7cee30843
Sep 10 17:49:28.790409 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.790385    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbg7n\" (UniqueName: \"kubernetes.io/projected/569add65-7e91-43b7-950b-2854e2caa6c5-kube-api-access-gbg7n\") pod \"authorino-59c974bd48-mgxqf\" (UID: \"569add65-7e91-43b7-950b-2854e2caa6c5\") " pod="kuadrant-system/authorino-59c974bd48-mgxqf"
Sep 10 17:49:28.890799 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.890756    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbg7n\" (UniqueName: \"kubernetes.io/projected/569add65-7e91-43b7-950b-2854e2caa6c5-kube-api-access-gbg7n\") pod \"authorino-59c974bd48-mgxqf\" (UID: \"569add65-7e91-43b7-950b-2854e2caa6c5\") " pod="kuadrant-system/authorino-59c974bd48-mgxqf"
Sep 10 17:49:28.900254 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:28.900217    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbg7n\" (UniqueName: \"kubernetes.io/projected/569add65-7e91-43b7-950b-2854e2caa6c5-kube-api-access-gbg7n\") pod \"authorino-59c974bd48-mgxqf\" (UID: \"569add65-7e91-43b7-950b-2854e2caa6c5\") " pod="kuadrant-system/authorino-59c974bd48-mgxqf"
Sep 10 17:49:29.027948 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:29.027865    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mgxqf"
Sep 10 17:49:29.269096 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:29.269061    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-ld75g" event={"ID":"12cca286-9d95-486f-9f8e-b0f4e530cb24","Type":"ContainerStarted","Data":"7b722a2c4547829e944d4093100c0a02b47c1fd32a04fd7b0bee6da7cee30843"}
Sep 10 17:49:29.297726 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:29.297644    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mgxqf"]
Sep 10 17:49:29.300868 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:49:29.300844    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod569add65_7e91_43b7_950b_2854e2caa6c5.slice/crio-76bb4338b8abc4f1145007a440172dd35c21124356c437742a6e0dcfd118ea2e WatchSource:0}: Error finding container 76bb4338b8abc4f1145007a440172dd35c21124356c437742a6e0dcfd118ea2e: Status 404 returned error can't find the container with id 76bb4338b8abc4f1145007a440172dd35c21124356c437742a6e0dcfd118ea2e
Sep 10 17:49:30.276133 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:30.276078    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mgxqf" event={"ID":"569add65-7e91-43b7-950b-2854e2caa6c5","Type":"ContainerStarted","Data":"76bb4338b8abc4f1145007a440172dd35c21124356c437742a6e0dcfd118ea2e"}
Sep 10 17:49:30.278319 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:30.278150    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-ld75g" event={"ID":"12cca286-9d95-486f-9f8e-b0f4e530cb24","Type":"ContainerStarted","Data":"197337c81f29b0999deca44c9d99dc6d1739f43340ffb477d08756faa739a054"}
Sep 10 17:49:30.278319 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:30.278280    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-ld75g"
Sep 10 17:49:30.296451 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:30.296401    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-ld75g" podStartSLOduration=1.3675953029999999 podStartE2EDuration="2.296380304s" totalImagesPullingTime="928.785001ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:30.295319402 +0000 UTC m=+494.786371223" watchObservedRunningTime="2026-09-10 17:49:30.296380304 +0000 UTC m=+494.787432112"
Sep 10 17:49:31.284469 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:31.284372    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mgxqf" event={"ID":"569add65-7e91-43b7-950b-2854e2caa6c5","Type":"ContainerStarted","Data":"ade8008b2e5e938c0f59aa5cbc667d8079852a92007453807ee3a0c845e1c707"}
Sep 10 17:49:31.300631 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:31.300583    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-mgxqf" podStartSLOduration=1.596133253 podStartE2EDuration="3.300567263s" totalImagesPullingTime="1.70443401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:31.300284184 +0000 UTC m=+495.791335993" watchObservedRunningTime="2026-09-10 17:49:31.300567263 +0000 UTC m=+495.791619069"
Sep 10 17:49:41.285353 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:41.285316    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-ld75g"
Sep 10 17:49:59.356231 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:59.356189    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mgxqf"]
Sep 10 17:49:59.356745 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:59.356424    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-mgxqf" podUID="569add65-7e91-43b7-950b-2854e2caa6c5" containerName="authorino" containerID="cri-o://ade8008b2e5e938c0f59aa5cbc667d8079852a92007453807ee3a0c845e1c707" gracePeriod=30
Sep 10 17:49:59.595507 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:59.595479    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mgxqf"
Sep 10 17:49:59.618549 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:59.618467    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbg7n\" (UniqueName: \"kubernetes.io/projected/569add65-7e91-43b7-950b-2854e2caa6c5-kube-api-access-gbg7n\") pod \"569add65-7e91-43b7-950b-2854e2caa6c5\" (UID: \"569add65-7e91-43b7-950b-2854e2caa6c5\") "
Sep 10 17:49:59.620630 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:59.620593    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/569add65-7e91-43b7-950b-2854e2caa6c5-kube-api-access-gbg7n" (OuterVolumeSpecName: "kube-api-access-gbg7n") pod "569add65-7e91-43b7-950b-2854e2caa6c5" (UID: "569add65-7e91-43b7-950b-2854e2caa6c5"). InnerVolumeSpecName "kube-api-access-gbg7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:49:59.719158 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:49:59.719116    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbg7n\" (UniqueName: \"kubernetes.io/projected/569add65-7e91-43b7-950b-2854e2caa6c5-kube-api-access-gbg7n\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:50:00.411449 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:00.411348    2549 generic.go:356] "Generic (PLEG): container finished" podID="569add65-7e91-43b7-950b-2854e2caa6c5" containerID="ade8008b2e5e938c0f59aa5cbc667d8079852a92007453807ee3a0c845e1c707" exitCode=0
Sep 10 17:50:00.411449 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:00.411390    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-mgxqf"
Sep 10 17:50:00.411449 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:00.411407    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mgxqf" event={"ID":"569add65-7e91-43b7-950b-2854e2caa6c5","Type":"ContainerDied","Data":"ade8008b2e5e938c0f59aa5cbc667d8079852a92007453807ee3a0c845e1c707"}
Sep 10 17:50:00.411959 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:00.411467    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-mgxqf" event={"ID":"569add65-7e91-43b7-950b-2854e2caa6c5","Type":"ContainerDied","Data":"76bb4338b8abc4f1145007a440172dd35c21124356c437742a6e0dcfd118ea2e"}
Sep 10 17:50:00.411959 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:00.411489    2549 scope.go:117] "RemoveContainer" containerID="ade8008b2e5e938c0f59aa5cbc667d8079852a92007453807ee3a0c845e1c707"
Sep 10 17:50:00.419716 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:00.419683    2549 scope.go:117] "RemoveContainer" containerID="ade8008b2e5e938c0f59aa5cbc667d8079852a92007453807ee3a0c845e1c707"
Sep 10 17:50:00.419968 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:50:00.419951    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ade8008b2e5e938c0f59aa5cbc667d8079852a92007453807ee3a0c845e1c707\": container with ID starting with ade8008b2e5e938c0f59aa5cbc667d8079852a92007453807ee3a0c845e1c707 not found: ID does not exist" containerID="ade8008b2e5e938c0f59aa5cbc667d8079852a92007453807ee3a0c845e1c707"
Sep 10 17:50:00.420015 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:00.419975    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ade8008b2e5e938c0f59aa5cbc667d8079852a92007453807ee3a0c845e1c707"} err="failed to get container status \"ade8008b2e5e938c0f59aa5cbc667d8079852a92007453807ee3a0c845e1c707\": rpc error: code = NotFound desc = could not find container \"ade8008b2e5e938c0f59aa5cbc667d8079852a92007453807ee3a0c845e1c707\": container with ID starting with ade8008b2e5e938c0f59aa5cbc667d8079852a92007453807ee3a0c845e1c707 not found: ID does not exist"
Sep 10 17:50:00.429347 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:00.429322    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mgxqf"]
Sep 10 17:50:00.430903 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:00.430885    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-mgxqf"]
Sep 10 17:50:02.028963 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:02.028925    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="569add65-7e91-43b7-950b-2854e2caa6c5" path="/var/lib/kubelet/pods/569add65-7e91-43b7-950b-2854e2caa6c5/volumes"
Sep 10 17:50:42.316012 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.315977    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"]
Sep 10 17:50:42.316454 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.316327    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="569add65-7e91-43b7-950b-2854e2caa6c5" containerName="authorino"
Sep 10 17:50:42.316454 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.316340    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="569add65-7e91-43b7-950b-2854e2caa6c5" containerName="authorino"
Sep 10 17:50:42.316454 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.316407    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="569add65-7e91-43b7-950b-2854e2caa6c5" containerName="authorino"
Sep 10 17:50:42.319237 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.319218    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:42.323970 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.323948    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 17:50:42.324129 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.324111    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 17:50:42.324286 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.324267    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:50:42.325219 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.325201    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 17:50:42.331633 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.331610    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"]
Sep 10 17:50:42.488808 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.488773    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-metrics-certs\") pod \"maas-api-54b46465b9-htdqx\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:42.488808 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.488820    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96jv5\" (UniqueName: \"kubernetes.io/projected/db4e1904-529b-4ff2-96ca-1346137636f8-kube-api-access-96jv5\") pod \"maas-api-54b46465b9-htdqx\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:42.489081 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.488893    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db4e1904-529b-4ff2-96ca-1346137636f8-openshift-service-ca\") pod \"maas-api-54b46465b9-htdqx\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:42.489081 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.489014    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-maas-api-tls\") pod \"maas-api-54b46465b9-htdqx\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:42.589498 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.589426    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-maas-api-tls\") pod \"maas-api-54b46465b9-htdqx\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:42.589498 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.589478    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-metrics-certs\") pod \"maas-api-54b46465b9-htdqx\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:42.589662 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.589515    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96jv5\" (UniqueName: \"kubernetes.io/projected/db4e1904-529b-4ff2-96ca-1346137636f8-kube-api-access-96jv5\") pod \"maas-api-54b46465b9-htdqx\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:42.589662 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.589559    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db4e1904-529b-4ff2-96ca-1346137636f8-openshift-service-ca\") pod \"maas-api-54b46465b9-htdqx\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:42.589662 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:50:42.589567    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 10 17:50:42.589662 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:50:42.589642    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 10 17:50:42.589833 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:50:42.589647    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-maas-api-tls podName:db4e1904-529b-4ff2-96ca-1346137636f8 nodeName:}" failed. No retries permitted until 2026-09-10 17:50:43.089625604 +0000 UTC m=+567.580677408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-maas-api-tls") pod "maas-api-54b46465b9-htdqx" (UID: "db4e1904-529b-4ff2-96ca-1346137636f8") : secret "maas-api-serving-cert" not found
Sep 10 17:50:42.589833 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:50:42.589735    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-metrics-certs podName:db4e1904-529b-4ff2-96ca-1346137636f8 nodeName:}" failed. No retries permitted until 2026-09-10 17:50:43.089717131 +0000 UTC m=+567.580768917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-metrics-certs") pod "maas-api-54b46465b9-htdqx" (UID: "db4e1904-529b-4ff2-96ca-1346137636f8") : secret "maas-api-metrics-tls" not found
Sep 10 17:50:42.590237 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.590220    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db4e1904-529b-4ff2-96ca-1346137636f8-openshift-service-ca\") pod \"maas-api-54b46465b9-htdqx\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:42.600817 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.600791    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96jv5\" (UniqueName: \"kubernetes.io/projected/db4e1904-529b-4ff2-96ca-1346137636f8-kube-api-access-96jv5\") pod \"maas-api-54b46465b9-htdqx\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:42.829886 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:42.829845    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:50:43.093760 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:43.093716    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-maas-api-tls\") pod \"maas-api-54b46465b9-htdqx\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:43.093947 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:43.093787    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-metrics-certs\") pod \"maas-api-54b46465b9-htdqx\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:43.096306 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:43.096274    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-maas-api-tls\") pod \"maas-api-54b46465b9-htdqx\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:43.096419 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:43.096274    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-metrics-certs\") pod \"maas-api-54b46465b9-htdqx\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") " pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:43.229381 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:43.229340    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:43.498527 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:43.498464    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"]
Sep 10 17:50:43.500711 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:50:43.500675    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb4e1904_529b_4ff2_96ca_1346137636f8.slice/crio-7482ca272f96c199722cefe5a69e6e451ab822ff31bde9d25cce2a8b3d6c2f8f WatchSource:0}: Error finding container 7482ca272f96c199722cefe5a69e6e451ab822ff31bde9d25cce2a8b3d6c2f8f: Status 404 returned error can't find the container with id 7482ca272f96c199722cefe5a69e6e451ab822ff31bde9d25cce2a8b3d6c2f8f
Sep 10 17:50:43.595438 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:43.595405    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx" event={"ID":"db4e1904-529b-4ff2-96ca-1346137636f8","Type":"ContainerStarted","Data":"7482ca272f96c199722cefe5a69e6e451ab822ff31bde9d25cce2a8b3d6c2f8f"}
Sep 10 17:50:46.611771 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:46.611730    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx" event={"ID":"db4e1904-529b-4ff2-96ca-1346137636f8","Type":"ContainerStarted","Data":"2c9243522a8c893151e7fddf5ee14ce100790d4ad09bc404ecec538374cae968"}
Sep 10 17:50:46.612168 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:46.611827    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:50:46.629444 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:46.629383    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx" podStartSLOduration=2.303016329 podStartE2EDuration="4.629363689s" totalImagesPullingTime="2.32634736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:50:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:50:46.628178841 +0000 UTC m=+571.119230662" watchObservedRunningTime="2026-09-10 17:50:46.629363689 +0000 UTC m=+571.120415497"
Sep 10 17:50:52.621967 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:50:52.621929    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 17:51:21.315174 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:51:21.315133    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:51:33.501227 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:51:33.501194    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:51:40.293363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:51:40.293277    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:51:51.999802 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:51:51.999771    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:52:20.322017 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.321985    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"]
Sep 10 17:52:20.325509 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.325485    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.329884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.329859    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 17:52:20.329994 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.329915    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 10 17:52:20.329994 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.329864    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 17:52:20.329994 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.329864    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6fj4v\""
Sep 10 17:52:20.336837 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.336815    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"]
Sep 10 17:52:20.434655 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.434618    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.434862 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.434664    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ffhd\" (UniqueName: \"kubernetes.io/projected/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-kube-api-access-8ffhd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.434862 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.434722    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.434862 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.434756    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.434862 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.434801    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.435017 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.434859    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.535624 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.535585    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.535830 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.535635    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ffhd\" (UniqueName: \"kubernetes.io/projected/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-kube-api-access-8ffhd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.535830 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.535665    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.535830 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.535707    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.535830 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.535753    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.535830 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.535786    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.536150 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.536067    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.536209 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.536180    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.536266 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.536239    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.537952 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.537931    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.538309 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.538291    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.545438 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.545410    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ffhd\" (UniqueName: \"kubernetes.io/projected/827c50a7-3b56-481e-8a22-6ae4cf6aaf5c-kube-api-access-8ffhd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4\" (UID: \"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.636373 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.636263    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:20.909141 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.909115    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"]
Sep 10 17:52:20.911812 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:52:20.911782    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod827c50a7_3b56_481e_8a22_6ae4cf6aaf5c.slice/crio-9a5934624cd97727ec1d1cba25c6d0bf0f2cd2859c18537b04b75e12e31d816d WatchSource:0}: Error finding container 9a5934624cd97727ec1d1cba25c6d0bf0f2cd2859c18537b04b75e12e31d816d: Status 404 returned error can't find the container with id 9a5934624cd97727ec1d1cba25c6d0bf0f2cd2859c18537b04b75e12e31d816d
Sep 10 17:52:20.913385 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:20.913364    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:52:21.008893 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:21.008858    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4" event={"ID":"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c","Type":"ContainerStarted","Data":"9a5934624cd97727ec1d1cba25c6d0bf0f2cd2859c18537b04b75e12e31d816d"}
Sep 10 17:52:22.002852 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:22.002820    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:52:23.019753 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:23.019711    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4" event={"ID":"827c50a7-3b56-481e-8a22-6ae4cf6aaf5c","Type":"ContainerStarted","Data":"3b4a8628933de82ac0fa664ae2282a0c14a7ca9382f02e06caf36d1658423bc7"}
Sep 10 17:52:23.039451 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:23.039390    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4" podStartSLOduration=1.982470426 podStartE2EDuration="3.03937435s" totalImagesPullingTime="1.056903924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:52:23.038266821 +0000 UTC m=+667.529318630" watchObservedRunningTime="2026-09-10 17:52:23.03937435 +0000 UTC m=+667.530426165"
Sep 10 17:52:30.110228 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:30.110183    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:52:30.636729 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:30.636669    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:30.636934 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:30.636768    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:30.648664 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:30.648632    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:31.065987 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:31.065954    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4"
Sep 10 17:52:39.109527 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.109493    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"]
Sep 10 17:52:39.113078 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.113051    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.116589 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.116568    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 17:52:39.134005 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.133978    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"]
Sep 10 17:52:39.193064 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.193025    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a54e3a8-887c-4f2c-9efe-6145059199f1-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.193064 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.193065    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a54e3a8-887c-4f2c-9efe-6145059199f1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.193269 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.193091    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qz6q\" (UniqueName: \"kubernetes.io/projected/4a54e3a8-887c-4f2c-9efe-6145059199f1-kube-api-access-7qz6q\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.193269 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.193169    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a54e3a8-887c-4f2c-9efe-6145059199f1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.193269 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.193224    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a54e3a8-887c-4f2c-9efe-6145059199f1-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.193269 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.193252    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a54e3a8-887c-4f2c-9efe-6145059199f1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.294408 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.294367    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qz6q\" (UniqueName: \"kubernetes.io/projected/4a54e3a8-887c-4f2c-9efe-6145059199f1-kube-api-access-7qz6q\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.294591 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.294424    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a54e3a8-887c-4f2c-9efe-6145059199f1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.294591 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.294459    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a54e3a8-887c-4f2c-9efe-6145059199f1-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.294591 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.294483    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a54e3a8-887c-4f2c-9efe-6145059199f1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.294591 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.294544    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a54e3a8-887c-4f2c-9efe-6145059199f1-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.294591 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.294568    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a54e3a8-887c-4f2c-9efe-6145059199f1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.294996 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.294964    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a54e3a8-887c-4f2c-9efe-6145059199f1-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.295064 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.294995    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a54e3a8-887c-4f2c-9efe-6145059199f1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.295064 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.295026    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a54e3a8-887c-4f2c-9efe-6145059199f1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.296933 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.296911    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a54e3a8-887c-4f2c-9efe-6145059199f1-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.297075 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.297055    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a54e3a8-887c-4f2c-9efe-6145059199f1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.302452 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.302427    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qz6q\" (UniqueName: \"kubernetes.io/projected/4a54e3a8-887c-4f2c-9efe-6145059199f1-kube-api-access-7qz6q\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9\" (UID: \"4a54e3a8-887c-4f2c-9efe-6145059199f1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.422660 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.422567    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:39.697056 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:39.697030    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"]
Sep 10 17:52:39.699623 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:52:39.699600    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a54e3a8_887c_4f2c_9efe_6145059199f1.slice/crio-2430d9c780eb461e907eb441b69f363a596e236c3cd2c481eb706561bf2456d7 WatchSource:0}: Error finding container 2430d9c780eb461e907eb441b69f363a596e236c3cd2c481eb706561bf2456d7: Status 404 returned error can't find the container with id 2430d9c780eb461e907eb441b69f363a596e236c3cd2c481eb706561bf2456d7
Sep 10 17:52:40.094263 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:40.094220    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9" event={"ID":"4a54e3a8-887c-4f2c-9efe-6145059199f1","Type":"ContainerStarted","Data":"0ee4e20389231b851b24882f56e6c25db36cae3a0b9195e93c7b20fa1e51ef8c"}
Sep 10 17:52:40.094406 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:40.094270    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9" event={"ID":"4a54e3a8-887c-4f2c-9efe-6145059199f1","Type":"ContainerStarted","Data":"2430d9c780eb461e907eb441b69f363a596e236c3cd2c481eb706561bf2456d7"}
Sep 10 17:52:40.120409 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:40.120352    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9" podStartSLOduration=0.923168534 podStartE2EDuration="1.120335276s" totalImagesPullingTime="197.166742ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:52:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:52:40.119672353 +0000 UTC m=+684.610724161" watchObservedRunningTime="2026-09-10 17:52:40.120335276 +0000 UTC m=+684.611387084"
Sep 10 17:52:41.405930 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:41.405896    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:52:49.423103 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:49.423058    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:49.423580 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:49.423129    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:49.435211 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:49.435180    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:52:50.147481 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:52:50.147448    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9"
Sep 10 17:54:07.164588 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:07.164547    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f7c475b8-9vwpt"]
Sep 10 17:54:07.167887 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:07.167870    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f7c475b8-9vwpt"
Sep 10 17:54:07.171867 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:07.171842    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8v6q4\""
Sep 10 17:54:07.172007 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:07.171878    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 17:54:07.175272 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:07.175251    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f7c475b8-9vwpt"]
Sep 10 17:54:07.354210 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:07.354168    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2be677c4-2c33-4fa0-b1a3-978dc1582f63-tls-cert\") pod \"authorino-f7c475b8-9vwpt\" (UID: \"2be677c4-2c33-4fa0-b1a3-978dc1582f63\") " pod="kuadrant-system/authorino-f7c475b8-9vwpt"
Sep 10 17:54:07.354397 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:07.354234    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67z4t\" (UniqueName: \"kubernetes.io/projected/2be677c4-2c33-4fa0-b1a3-978dc1582f63-kube-api-access-67z4t\") pod \"authorino-f7c475b8-9vwpt\" (UID: \"2be677c4-2c33-4fa0-b1a3-978dc1582f63\") " pod="kuadrant-system/authorino-f7c475b8-9vwpt"
Sep 10 17:54:07.454980 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:07.454946    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67z4t\" (UniqueName: \"kubernetes.io/projected/2be677c4-2c33-4fa0-b1a3-978dc1582f63-kube-api-access-67z4t\") pod \"authorino-f7c475b8-9vwpt\" (UID: \"2be677c4-2c33-4fa0-b1a3-978dc1582f63\") " pod="kuadrant-system/authorino-f7c475b8-9vwpt"
Sep 10 17:54:07.455154 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:07.455059    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2be677c4-2c33-4fa0-b1a3-978dc1582f63-tls-cert\") pod \"authorino-f7c475b8-9vwpt\" (UID: \"2be677c4-2c33-4fa0-b1a3-978dc1582f63\") " pod="kuadrant-system/authorino-f7c475b8-9vwpt"
Sep 10 17:54:07.457617 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:07.457590    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2be677c4-2c33-4fa0-b1a3-978dc1582f63-tls-cert\") pod \"authorino-f7c475b8-9vwpt\" (UID: \"2be677c4-2c33-4fa0-b1a3-978dc1582f63\") " pod="kuadrant-system/authorino-f7c475b8-9vwpt"
Sep 10 17:54:07.463314 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:07.463281    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67z4t\" (UniqueName: \"kubernetes.io/projected/2be677c4-2c33-4fa0-b1a3-978dc1582f63-kube-api-access-67z4t\") pod \"authorino-f7c475b8-9vwpt\" (UID: \"2be677c4-2c33-4fa0-b1a3-978dc1582f63\") " pod="kuadrant-system/authorino-f7c475b8-9vwpt"
Sep 10 17:54:07.477280 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:07.477250    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f7c475b8-9vwpt"
Sep 10 17:54:07.747896 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:07.747836    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f7c475b8-9vwpt"]
Sep 10 17:54:07.750274 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:54:07.750236    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2be677c4_2c33_4fa0_b1a3_978dc1582f63.slice/crio-ba891a923ffe53f1f8056186d0c08d6fb13cf5ec337db384fffc9b821c6b9d5a WatchSource:0}: Error finding container ba891a923ffe53f1f8056186d0c08d6fb13cf5ec337db384fffc9b821c6b9d5a: Status 404 returned error can't find the container with id ba891a923ffe53f1f8056186d0c08d6fb13cf5ec337db384fffc9b821c6b9d5a
Sep 10 17:54:08.480656 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:08.480617    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f7c475b8-9vwpt" event={"ID":"2be677c4-2c33-4fa0-b1a3-978dc1582f63","Type":"ContainerStarted","Data":"2442ec8b1f64f8bbd2480d3a8435cf00cd50a4cc8c25dd08250a03246a9ec433"}
Sep 10 17:54:08.481199 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:08.480665    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f7c475b8-9vwpt" event={"ID":"2be677c4-2c33-4fa0-b1a3-978dc1582f63","Type":"ContainerStarted","Data":"ba891a923ffe53f1f8056186d0c08d6fb13cf5ec337db384fffc9b821c6b9d5a"}
Sep 10 17:54:08.500035 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:08.499982    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-f7c475b8-9vwpt" podStartSLOduration=1.07374807 podStartE2EDuration="1.499965402s" totalImagesPullingTime="426.217332ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:08.499077463 +0000 UTC m=+772.990129272" watchObservedRunningTime="2026-09-10 17:54:08.499965402 +0000 UTC m=+772.991017210"
Sep 10 17:54:29.707066 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:29.707025    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:54:30.693100 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.693067    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"]
Sep 10 17:54:30.696525 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.696503    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:30.700503 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.700480    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-c34c13-serving-cert\""
Sep 10 17:54:30.700662 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.700641    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-n56s4\""
Sep 10 17:54:30.707204 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.707179    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"]
Sep 10 17:54:30.755089 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.755054    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbae3a46-3569-48c1-aa82-c628fd07651f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:30.755258 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.755097    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbae3a46-3569-48c1-aa82-c628fd07651f-maas-api-tls\") pod \"maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:30.755258 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.755177    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlxxv\" (UniqueName: \"kubernetes.io/projected/dbae3a46-3569-48c1-aa82-c628fd07651f-kube-api-access-zlxxv\") pod \"maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:30.755258 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.755243    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbae3a46-3569-48c1-aa82-c628fd07651f-metrics-certs\") pod \"maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:30.855818 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.855776    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbae3a46-3569-48c1-aa82-c628fd07651f-maas-api-tls\") pod \"maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:30.855948 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.855839    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlxxv\" (UniqueName: \"kubernetes.io/projected/dbae3a46-3569-48c1-aa82-c628fd07651f-kube-api-access-zlxxv\") pod \"maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:30.855948 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.855887    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbae3a46-3569-48c1-aa82-c628fd07651f-metrics-certs\") pod \"maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:30.855948 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.855915    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbae3a46-3569-48c1-aa82-c628fd07651f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:30.855948 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:54:30.855934    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-serving-cert: secret "maas-api-e2e-inf-a-c34c13-serving-cert" not found
Sep 10 17:54:30.856080 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:54:30.856022    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbae3a46-3569-48c1-aa82-c628fd07651f-maas-api-tls podName:dbae3a46-3569-48c1-aa82-c628fd07651f nodeName:}" failed. No retries permitted until 2026-09-10 17:54:31.355998851 +0000 UTC m=+795.847050651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dbae3a46-3569-48c1-aa82-c628fd07651f-maas-api-tls") pod "maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z" (UID: "dbae3a46-3569-48c1-aa82-c628fd07651f") : secret "maas-api-e2e-inf-a-c34c13-serving-cert" not found
Sep 10 17:54:30.856634 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.856618    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbae3a46-3569-48c1-aa82-c628fd07651f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:30.858277 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.858257    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbae3a46-3569-48c1-aa82-c628fd07651f-metrics-certs\") pod \"maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:30.864166 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:30.864142    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlxxv\" (UniqueName: \"kubernetes.io/projected/dbae3a46-3569-48c1-aa82-c628fd07651f-kube-api-access-zlxxv\") pod \"maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:31.359821 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:31.359769    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbae3a46-3569-48c1-aa82-c628fd07651f-maas-api-tls\") pod \"maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:31.362347 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:31.362304    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbae3a46-3569-48c1-aa82-c628fd07651f-maas-api-tls\") pod \"maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:31.607776 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:31.607737    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:31.887738 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:31.887708    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"]
Sep 10 17:54:31.890960 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:54:31.890927    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbae3a46_3569_48c1_aa82_c628fd07651f.slice/crio-e62406cf7f35e222d03f1689ddd682b66ca33a21abc415f9707fc04d9bfe0805 WatchSource:0}: Error finding container e62406cf7f35e222d03f1689ddd682b66ca33a21abc415f9707fc04d9bfe0805: Status 404 returned error can't find the container with id e62406cf7f35e222d03f1689ddd682b66ca33a21abc415f9707fc04d9bfe0805
Sep 10 17:54:32.588578 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:32.588487    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z" event={"ID":"dbae3a46-3569-48c1-aa82-c628fd07651f","Type":"ContainerStarted","Data":"2f46e3c1019304aa65e5c222c9392ff70824e89a38764824d141236ab744cb3f"}
Sep 10 17:54:32.588578 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:32.588553    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z" event={"ID":"dbae3a46-3569-48c1-aa82-c628fd07651f","Type":"ContainerStarted","Data":"e62406cf7f35e222d03f1689ddd682b66ca33a21abc415f9707fc04d9bfe0805"}
Sep 10 17:54:32.588808 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:32.588585    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:32.606328 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:32.606273    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z" podStartSLOduration=2.248817266 podStartE2EDuration="2.606255159s" totalImagesPullingTime="357.437893ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:32.605027946 +0000 UTC m=+797.096079753" watchObservedRunningTime="2026-09-10 17:54:32.606255159 +0000 UTC m=+797.097306966"
Sep 10 17:54:38.128617 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.128546    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"]
Sep 10 17:54:38.132169 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.132146    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:38.135020 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.134995    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-007ef522-serving-cert\""
Sep 10 17:54:38.140982 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.140952    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"]
Sep 10 17:54:38.227437 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.227395    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6p7z\" (UniqueName: \"kubernetes.io/projected/86ad2f62-da17-4d52-9097-b539e89fcff3-kube-api-access-x6p7z\") pod \"maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:38.227594 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.227513    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86ad2f62-da17-4d52-9097-b539e89fcff3-openshift-service-ca\") pod \"maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:38.227650 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.227612    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86ad2f62-da17-4d52-9097-b539e89fcff3-metrics-certs\") pod \"maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:38.227724 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.227677    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86ad2f62-da17-4d52-9097-b539e89fcff3-maas-api-tls\") pod \"maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:38.328451 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.328417    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86ad2f62-da17-4d52-9097-b539e89fcff3-metrics-certs\") pod \"maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:38.328609 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.328477    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86ad2f62-da17-4d52-9097-b539e89fcff3-maas-api-tls\") pod \"maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:38.328609 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.328517    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6p7z\" (UniqueName: \"kubernetes.io/projected/86ad2f62-da17-4d52-9097-b539e89fcff3-kube-api-access-x6p7z\") pod \"maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:38.328609 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.328565    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86ad2f62-da17-4d52-9097-b539e89fcff3-openshift-service-ca\") pod \"maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:38.329311 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.329286    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86ad2f62-da17-4d52-9097-b539e89fcff3-openshift-service-ca\") pod \"maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:38.331110 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.331081    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86ad2f62-da17-4d52-9097-b539e89fcff3-maas-api-tls\") pod \"maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:38.331205 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.331121    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86ad2f62-da17-4d52-9097-b539e89fcff3-metrics-certs\") pod \"maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:38.338889 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.338858    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6p7z\" (UniqueName: \"kubernetes.io/projected/86ad2f62-da17-4d52-9097-b539e89fcff3-kube-api-access-x6p7z\") pod \"maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:38.443943 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.443839    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:38.598655 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.598625    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:54:38.745088 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:38.745052    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"]
Sep 10 17:54:38.748907 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:54:38.748879    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86ad2f62_da17_4d52_9097_b539e89fcff3.slice/crio-5f8fd467117d622a8141333b06a3dc798127dd312088e39f69bb1839627cb90c WatchSource:0}: Error finding container 5f8fd467117d622a8141333b06a3dc798127dd312088e39f69bb1839627cb90c: Status 404 returned error can't find the container with id 5f8fd467117d622a8141333b06a3dc798127dd312088e39f69bb1839627cb90c
Sep 10 17:54:39.319803 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.319677    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"]
Sep 10 17:54:39.323525 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.323493    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:39.325980 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.325944    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"]
Sep 10 17:54:39.326459 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.326433    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-bcdd7c-serving-cert\""
Sep 10 17:54:39.437638 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.437601    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62539a9b-5ede-4aab-91f8-4dd7242598ac-maas-api-tls\") pod \"maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:39.437817 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.437650    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhw4m\" (UniqueName: \"kubernetes.io/projected/62539a9b-5ede-4aab-91f8-4dd7242598ac-kube-api-access-qhw4m\") pod \"maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:39.437817 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.437728    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62539a9b-5ede-4aab-91f8-4dd7242598ac-openshift-service-ca\") pod \"maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:39.437817 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.437780    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62539a9b-5ede-4aab-91f8-4dd7242598ac-metrics-certs\") pod \"maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:39.538562 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.538518    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62539a9b-5ede-4aab-91f8-4dd7242598ac-maas-api-tls\") pod \"maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:39.538768 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.538573    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhw4m\" (UniqueName: \"kubernetes.io/projected/62539a9b-5ede-4aab-91f8-4dd7242598ac-kube-api-access-qhw4m\") pod \"maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:39.538768 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.538603    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62539a9b-5ede-4aab-91f8-4dd7242598ac-openshift-service-ca\") pod \"maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:39.538768 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.538626    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62539a9b-5ede-4aab-91f8-4dd7242598ac-metrics-certs\") pod \"maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:39.538768 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:54:39.538670    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-serving-cert: secret "maas-api-e2e-autores-bcdd7c-serving-cert" not found
Sep 10 17:54:39.538768 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:54:39.538762    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62539a9b-5ede-4aab-91f8-4dd7242598ac-maas-api-tls podName:62539a9b-5ede-4aab-91f8-4dd7242598ac nodeName:}" failed. No retries permitted until 2026-09-10 17:54:40.038744061 +0000 UTC m=+804.529795851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/62539a9b-5ede-4aab-91f8-4dd7242598ac-maas-api-tls") pod "maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz" (UID: "62539a9b-5ede-4aab-91f8-4dd7242598ac") : secret "maas-api-e2e-autores-bcdd7c-serving-cert" not found
Sep 10 17:54:39.539371 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.539347    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62539a9b-5ede-4aab-91f8-4dd7242598ac-openshift-service-ca\") pod \"maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:39.541122 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.541104    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62539a9b-5ede-4aab-91f8-4dd7242598ac-metrics-certs\") pod \"maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:39.548510 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.548489    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhw4m\" (UniqueName: \"kubernetes.io/projected/62539a9b-5ede-4aab-91f8-4dd7242598ac-kube-api-access-qhw4m\") pod \"maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:39.622015 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.621907    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj" event={"ID":"86ad2f62-da17-4d52-9097-b539e89fcff3","Type":"ContainerStarted","Data":"c2efe18344395640146f9b65ad0630ee2a29559099a4ec1ecc01ed84e560d17a"}
Sep 10 17:54:39.622015 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.621973    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj" event={"ID":"86ad2f62-da17-4d52-9097-b539e89fcff3","Type":"ContainerStarted","Data":"5f8fd467117d622a8141333b06a3dc798127dd312088e39f69bb1839627cb90c"}
Sep 10 17:54:39.622015 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.622003    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:39.643091 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:39.643034    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj" podStartSLOduration=1.345404764 podStartE2EDuration="1.643013986s" totalImagesPullingTime="297.609222ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:39.641321479 +0000 UTC m=+804.132373289" watchObservedRunningTime="2026-09-10 17:54:39.643013986 +0000 UTC m=+804.134065855"
Sep 10 17:54:40.044747 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:40.044685    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62539a9b-5ede-4aab-91f8-4dd7242598ac-maas-api-tls\") pod \"maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:40.047280 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:40.047253    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62539a9b-5ede-4aab-91f8-4dd7242598ac-maas-api-tls\") pod \"maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:40.248869 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:40.248831    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:40.521635 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:40.521609    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"]
Sep 10 17:54:40.524103 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:54:40.524074    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62539a9b_5ede_4aab_91f8_4dd7242598ac.slice/crio-880e923351a54f96924bd5d06811cc55a76c36c0ae0118c1978d42a309459b82 WatchSource:0}: Error finding container 880e923351a54f96924bd5d06811cc55a76c36c0ae0118c1978d42a309459b82: Status 404 returned error can't find the container with id 880e923351a54f96924bd5d06811cc55a76c36c0ae0118c1978d42a309459b82
Sep 10 17:54:40.628574 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:40.628484    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz" event={"ID":"62539a9b-5ede-4aab-91f8-4dd7242598ac","Type":"ContainerStarted","Data":"880e923351a54f96924bd5d06811cc55a76c36c0ae0118c1978d42a309459b82"}
Sep 10 17:54:41.635081 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:41.635046    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz" event={"ID":"62539a9b-5ede-4aab-91f8-4dd7242598ac","Type":"ContainerStarted","Data":"5ce5de4f63927057dc8fd7aba952f0d4ab545fc5869eecf114664a831a565992"}
Sep 10 17:54:41.635522 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:41.635111    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:54:41.654262 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:41.654207    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz" podStartSLOduration=2.378271145 podStartE2EDuration="2.654189401s" totalImagesPullingTime="275.918256ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:41.651864194 +0000 UTC m=+806.142916015" watchObservedRunningTime="2026-09-10 17:54:41.654189401 +0000 UTC m=+806.145241209"
Sep 10 17:54:45.006151 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:45.006101    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:54:45.633853 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:45.633824    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:54:47.652964 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:54:47.652921    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:55:00.290093 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.290056    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"]
Sep 10 17:55:00.294629 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.294602    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:00.297928 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.297900    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-18490e-serving-cert\""
Sep 10 17:55:00.303668 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.303572    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"]
Sep 10 17:55:00.333972 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.333935    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/332038d8-86bf-49e3-b5f2-e79743242879-maas-api-tls\") pod \"maas-api-e2e-inf-b-18490e-789d779469-szw8m\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:00.334142 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.333991    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/332038d8-86bf-49e3-b5f2-e79743242879-openshift-service-ca\") pod \"maas-api-e2e-inf-b-18490e-789d779469-szw8m\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:00.334142 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.334050    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332038d8-86bf-49e3-b5f2-e79743242879-metrics-certs\") pod \"maas-api-e2e-inf-b-18490e-789d779469-szw8m\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:00.334233 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.334131    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58vpj\" (UniqueName: \"kubernetes.io/projected/332038d8-86bf-49e3-b5f2-e79743242879-kube-api-access-58vpj\") pod \"maas-api-e2e-inf-b-18490e-789d779469-szw8m\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:00.435418 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.435376    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/332038d8-86bf-49e3-b5f2-e79743242879-openshift-service-ca\") pod \"maas-api-e2e-inf-b-18490e-789d779469-szw8m\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:00.435600 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.435460    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332038d8-86bf-49e3-b5f2-e79743242879-metrics-certs\") pod \"maas-api-e2e-inf-b-18490e-789d779469-szw8m\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:00.435600 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.435527    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58vpj\" (UniqueName: \"kubernetes.io/projected/332038d8-86bf-49e3-b5f2-e79743242879-kube-api-access-58vpj\") pod \"maas-api-e2e-inf-b-18490e-789d779469-szw8m\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:00.435600 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.435579    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/332038d8-86bf-49e3-b5f2-e79743242879-maas-api-tls\") pod \"maas-api-e2e-inf-b-18490e-789d779469-szw8m\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:00.435774 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:55:00.435715    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-serving-cert: secret "maas-api-e2e-inf-b-18490e-serving-cert" not found
Sep 10 17:55:00.435823 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:55:00.435792    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/332038d8-86bf-49e3-b5f2-e79743242879-maas-api-tls podName:332038d8-86bf-49e3-b5f2-e79743242879 nodeName:}" failed. No retries permitted until 2026-09-10 17:55:00.935769475 +0000 UTC m=+825.426821280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/332038d8-86bf-49e3-b5f2-e79743242879-maas-api-tls") pod "maas-api-e2e-inf-b-18490e-789d779469-szw8m" (UID: "332038d8-86bf-49e3-b5f2-e79743242879") : secret "maas-api-e2e-inf-b-18490e-serving-cert" not found
Sep 10 17:55:00.436241 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.436216    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/332038d8-86bf-49e3-b5f2-e79743242879-openshift-service-ca\") pod \"maas-api-e2e-inf-b-18490e-789d779469-szw8m\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:00.438054 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.438027    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332038d8-86bf-49e3-b5f2-e79743242879-metrics-certs\") pod \"maas-api-e2e-inf-b-18490e-789d779469-szw8m\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:00.444951 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.444929    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58vpj\" (UniqueName: \"kubernetes.io/projected/332038d8-86bf-49e3-b5f2-e79743242879-kube-api-access-58vpj\") pod \"maas-api-e2e-inf-b-18490e-789d779469-szw8m\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:00.939437 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.939393    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/332038d8-86bf-49e3-b5f2-e79743242879-maas-api-tls\") pod \"maas-api-e2e-inf-b-18490e-789d779469-szw8m\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:00.941971 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:00.941942    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/332038d8-86bf-49e3-b5f2-e79743242879-maas-api-tls\") pod \"maas-api-e2e-inf-b-18490e-789d779469-szw8m\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:01.208887 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:01.208851    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:01.484745 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:01.484719    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"]
Sep 10 17:55:01.742631 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:01.742546    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m" event={"ID":"332038d8-86bf-49e3-b5f2-e79743242879","Type":"ContainerStarted","Data":"d4e19292395111c866a1dfeab9647ad6b47245395bead1d3f1c3470b8f11343f"}
Sep 10 17:55:02.748617 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:02.748574    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m" event={"ID":"332038d8-86bf-49e3-b5f2-e79743242879","Type":"ContainerStarted","Data":"48bd1b64f3cc79808eb0b512bce601b6832a23087b2cab926954f984bbc0a82e"}
Sep 10 17:55:02.749025 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:02.748678    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:02.768777 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:02.768730    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m" podStartSLOduration=2.42365978 podStartE2EDuration="2.768714109s" totalImagesPullingTime="345.054329ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:55:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:55:02.765804656 +0000 UTC m=+827.256856465" watchObservedRunningTime="2026-09-10 17:55:02.768714109 +0000 UTC m=+827.259765907"
Sep 10 17:55:06.105946 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:06.105903    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:55:08.758654 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:08.758620    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:55:18.996470 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:18.996434    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"]
Sep 10 17:55:18.996998 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:18.996659    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj" podUID="86ad2f62-da17-4d52-9097-b539e89fcff3" containerName="maas-api" containerID="cri-o://c2efe18344395640146f9b65ad0630ee2a29559099a4ec1ecc01ed84e560d17a" gracePeriod=30
Sep 10 17:55:19.246800 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.246740    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:55:19.315750 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.315720    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86ad2f62-da17-4d52-9097-b539e89fcff3-metrics-certs\") pod \"86ad2f62-da17-4d52-9097-b539e89fcff3\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") "
Sep 10 17:55:19.315903 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.315802    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86ad2f62-da17-4d52-9097-b539e89fcff3-maas-api-tls\") pod \"86ad2f62-da17-4d52-9097-b539e89fcff3\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") "
Sep 10 17:55:19.315903 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.315830    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86ad2f62-da17-4d52-9097-b539e89fcff3-openshift-service-ca\") pod \"86ad2f62-da17-4d52-9097-b539e89fcff3\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") "
Sep 10 17:55:19.315903 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.315877    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6p7z\" (UniqueName: \"kubernetes.io/projected/86ad2f62-da17-4d52-9097-b539e89fcff3-kube-api-access-x6p7z\") pod \"86ad2f62-da17-4d52-9097-b539e89fcff3\" (UID: \"86ad2f62-da17-4d52-9097-b539e89fcff3\") "
Sep 10 17:55:19.316252 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.316215    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86ad2f62-da17-4d52-9097-b539e89fcff3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "86ad2f62-da17-4d52-9097-b539e89fcff3" (UID: "86ad2f62-da17-4d52-9097-b539e89fcff3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:55:19.317957 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.317932    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86ad2f62-da17-4d52-9097-b539e89fcff3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "86ad2f62-da17-4d52-9097-b539e89fcff3" (UID: "86ad2f62-da17-4d52-9097-b539e89fcff3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:55:19.318039 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.318004    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86ad2f62-da17-4d52-9097-b539e89fcff3-kube-api-access-x6p7z" (OuterVolumeSpecName: "kube-api-access-x6p7z") pod "86ad2f62-da17-4d52-9097-b539e89fcff3" (UID: "86ad2f62-da17-4d52-9097-b539e89fcff3"). InnerVolumeSpecName "kube-api-access-x6p7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:55:19.318039 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.318011    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86ad2f62-da17-4d52-9097-b539e89fcff3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "86ad2f62-da17-4d52-9097-b539e89fcff3" (UID: "86ad2f62-da17-4d52-9097-b539e89fcff3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:55:19.417218 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.417183    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86ad2f62-da17-4d52-9097-b539e89fcff3-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:55:19.417218 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.417211    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86ad2f62-da17-4d52-9097-b539e89fcff3-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:55:19.417218 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.417220    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86ad2f62-da17-4d52-9097-b539e89fcff3-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:55:19.417432 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.417229    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6p7z\" (UniqueName: \"kubernetes.io/projected/86ad2f62-da17-4d52-9097-b539e89fcff3-kube-api-access-x6p7z\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:55:19.828553 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.828518    2549 generic.go:356] "Generic (PLEG): container finished" podID="86ad2f62-da17-4d52-9097-b539e89fcff3" containerID="c2efe18344395640146f9b65ad0630ee2a29559099a4ec1ecc01ed84e560d17a" exitCode=0
Sep 10 17:55:19.828754 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.828563    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"
Sep 10 17:55:19.828754 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.828592    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj" event={"ID":"86ad2f62-da17-4d52-9097-b539e89fcff3","Type":"ContainerDied","Data":"c2efe18344395640146f9b65ad0630ee2a29559099a4ec1ecc01ed84e560d17a"}
Sep 10 17:55:19.828754 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.828632    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj" event={"ID":"86ad2f62-da17-4d52-9097-b539e89fcff3","Type":"ContainerDied","Data":"5f8fd467117d622a8141333b06a3dc798127dd312088e39f69bb1839627cb90c"}
Sep 10 17:55:19.828754 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.828654    2549 scope.go:117] "RemoveContainer" containerID="c2efe18344395640146f9b65ad0630ee2a29559099a4ec1ecc01ed84e560d17a"
Sep 10 17:55:19.837582 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.837564    2549 scope.go:117] "RemoveContainer" containerID="c2efe18344395640146f9b65ad0630ee2a29559099a4ec1ecc01ed84e560d17a"
Sep 10 17:55:19.837867 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:55:19.837843    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2efe18344395640146f9b65ad0630ee2a29559099a4ec1ecc01ed84e560d17a\": container with ID starting with c2efe18344395640146f9b65ad0630ee2a29559099a4ec1ecc01ed84e560d17a not found: ID does not exist" containerID="c2efe18344395640146f9b65ad0630ee2a29559099a4ec1ecc01ed84e560d17a"
Sep 10 17:55:19.837909 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.837876    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2efe18344395640146f9b65ad0630ee2a29559099a4ec1ecc01ed84e560d17a"} err="failed to get container status \"c2efe18344395640146f9b65ad0630ee2a29559099a4ec1ecc01ed84e560d17a\": rpc error: code = NotFound desc = could not find container \"c2efe18344395640146f9b65ad0630ee2a29559099a4ec1ecc01ed84e560d17a\": container with ID starting with c2efe18344395640146f9b65ad0630ee2a29559099a4ec1ecc01ed84e560d17a not found: ID does not exist"
Sep 10 17:55:19.855668 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.855641    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"]
Sep 10 17:55:19.861840 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:19.861815    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-007ef522-c89f9b8f9-8r5vj"]
Sep 10 17:55:20.034606 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:20.034567    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86ad2f62-da17-4d52-9097-b539e89fcff3" path="/var/lib/kubelet/pods/86ad2f62-da17-4d52-9097-b539e89fcff3/volumes"
Sep 10 17:55:23.715437 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.715402    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"]
Sep 10 17:55:23.716027 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.716005    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86ad2f62-da17-4d52-9097-b539e89fcff3" containerName="maas-api"
Sep 10 17:55:23.716075 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.716031    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="86ad2f62-da17-4d52-9097-b539e89fcff3" containerName="maas-api"
Sep 10 17:55:23.716145 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.716133    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="86ad2f62-da17-4d52-9097-b539e89fcff3" containerName="maas-api"
Sep 10 17:55:23.721086 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.721027    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:23.724094 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.724073    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-826cc2b1-serving-cert\""
Sep 10 17:55:23.731818 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.731794    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"]
Sep 10 17:55:23.857995 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.857961    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0cb9dd6-b140-4616-8905-21f89f14a61c-maas-api-tls\") pod \"maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:23.858156 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.858015    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0cb9dd6-b140-4616-8905-21f89f14a61c-openshift-service-ca\") pod \"maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:23.858156 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.858039    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr8fq\" (UniqueName: \"kubernetes.io/projected/d0cb9dd6-b140-4616-8905-21f89f14a61c-kube-api-access-kr8fq\") pod \"maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:23.858156 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.858103    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0cb9dd6-b140-4616-8905-21f89f14a61c-metrics-certs\") pod \"maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:23.958977 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.958937    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0cb9dd6-b140-4616-8905-21f89f14a61c-openshift-service-ca\") pod \"maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:23.958977 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.958982    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr8fq\" (UniqueName: \"kubernetes.io/projected/d0cb9dd6-b140-4616-8905-21f89f14a61c-kube-api-access-kr8fq\") pod \"maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:23.959232 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.959032    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0cb9dd6-b140-4616-8905-21f89f14a61c-metrics-certs\") pod \"maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:23.959232 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.959095    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0cb9dd6-b140-4616-8905-21f89f14a61c-maas-api-tls\") pod \"maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:23.959232 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:55:23.959202    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-serving-cert: secret "maas-api-e2e-migrate-826cc2b1-serving-cert" not found
Sep 10 17:55:23.959378 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:55:23.959270    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0cb9dd6-b140-4616-8905-21f89f14a61c-maas-api-tls podName:d0cb9dd6-b140-4616-8905-21f89f14a61c nodeName:}" failed. No retries permitted until 2026-09-10 17:55:24.459248497 +0000 UTC m=+848.950300301 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d0cb9dd6-b140-4616-8905-21f89f14a61c-maas-api-tls") pod "maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62" (UID: "d0cb9dd6-b140-4616-8905-21f89f14a61c") : secret "maas-api-e2e-migrate-826cc2b1-serving-cert" not found
Sep 10 17:55:23.959744 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.959718    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0cb9dd6-b140-4616-8905-21f89f14a61c-openshift-service-ca\") pod \"maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:23.961472 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.961454    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0cb9dd6-b140-4616-8905-21f89f14a61c-metrics-certs\") pod \"maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:23.967718 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:23.967658    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr8fq\" (UniqueName: \"kubernetes.io/projected/d0cb9dd6-b140-4616-8905-21f89f14a61c-kube-api-access-kr8fq\") pod \"maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:24.464481 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:24.464448    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0cb9dd6-b140-4616-8905-21f89f14a61c-maas-api-tls\") pod \"maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:24.466861 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:24.466833    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0cb9dd6-b140-4616-8905-21f89f14a61c-maas-api-tls\") pod \"maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:24.634571 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:24.634532    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:24.907285 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:24.907259    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"]
Sep 10 17:55:24.909093 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:55:24.909066    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0cb9dd6_b140_4616_8905_21f89f14a61c.slice/crio-75f489be0f596ca9f2708887e20ba88beb2524151a0ca892f1af0b6e70607b84 WatchSource:0}: Error finding container 75f489be0f596ca9f2708887e20ba88beb2524151a0ca892f1af0b6e70607b84: Status 404 returned error can't find the container with id 75f489be0f596ca9f2708887e20ba88beb2524151a0ca892f1af0b6e70607b84
Sep 10 17:55:25.860810 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:25.860774    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62" event={"ID":"d0cb9dd6-b140-4616-8905-21f89f14a61c","Type":"ContainerStarted","Data":"3a9a369a035832bf72d00f74c62d3d80f557a8c39242594e8a1233652d7aaf0c"}
Sep 10 17:55:25.860980 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:25.860856    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62" event={"ID":"d0cb9dd6-b140-4616-8905-21f89f14a61c","Type":"ContainerStarted","Data":"75f489be0f596ca9f2708887e20ba88beb2524151a0ca892f1af0b6e70607b84"}
Sep 10 17:55:25.860980 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:25.860900    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:25.880535 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:25.880488    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62" podStartSLOduration=2.575885916 podStartE2EDuration="2.880473956s" totalImagesPullingTime="304.58804ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:55:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:55:25.879196521 +0000 UTC m=+850.370248329" watchObservedRunningTime="2026-09-10 17:55:25.880473956 +0000 UTC m=+850.371525819"
Sep 10 17:55:26.098365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:26.098330    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:55:31.870364 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:31.870330    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:43.705619 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:43.705579    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:55:54.940724 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:54.940672    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"]
Sep 10 17:55:54.941214 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:54.940904    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62" podUID="d0cb9dd6-b140-4616-8905-21f89f14a61c" containerName="maas-api" containerID="cri-o://3a9a369a035832bf72d00f74c62d3d80f557a8c39242594e8a1233652d7aaf0c" gracePeriod=30
Sep 10 17:55:55.191465 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:55.191405    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:55.241227 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:55.241186    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0cb9dd6-b140-4616-8905-21f89f14a61c-openshift-service-ca\") pod \"d0cb9dd6-b140-4616-8905-21f89f14a61c\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") "
Sep 10 17:55:55.241416 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:55.241238    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0cb9dd6-b140-4616-8905-21f89f14a61c-metrics-certs\") pod \"d0cb9dd6-b140-4616-8905-21f89f14a61c\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") "
Sep 10 17:55:55.241416 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:55.241267    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kr8fq\" (UniqueName: \"kubernetes.io/projected/d0cb9dd6-b140-4616-8905-21f89f14a61c-kube-api-access-kr8fq\") pod \"d0cb9dd6-b140-4616-8905-21f89f14a61c\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") "
Sep 10 17:55:55.241540 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:55.241425    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0cb9dd6-b140-4616-8905-21f89f14a61c-maas-api-tls\") pod \"d0cb9dd6-b140-4616-8905-21f89f14a61c\" (UID: \"d0cb9dd6-b140-4616-8905-21f89f14a61c\") "
Sep 10 17:55:55.241595 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:55.241567    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0cb9dd6-b140-4616-8905-21f89f14a61c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d0cb9dd6-b140-4616-8905-21f89f14a61c" (UID: "d0cb9dd6-b140-4616-8905-21f89f14a61c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:55:55.241741 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:55.241715    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0cb9dd6-b140-4616-8905-21f89f14a61c-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:55:55.243513 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:55.243490    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0cb9dd6-b140-4616-8905-21f89f14a61c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d0cb9dd6-b140-4616-8905-21f89f14a61c" (UID: "d0cb9dd6-b140-4616-8905-21f89f14a61c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:55:55.243881 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:55.243860    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0cb9dd6-b140-4616-8905-21f89f14a61c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d0cb9dd6-b140-4616-8905-21f89f14a61c" (UID: "d0cb9dd6-b140-4616-8905-21f89f14a61c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:55:55.244003 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:55.243978    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0cb9dd6-b140-4616-8905-21f89f14a61c-kube-api-access-kr8fq" (OuterVolumeSpecName: "kube-api-access-kr8fq") pod "d0cb9dd6-b140-4616-8905-21f89f14a61c" (UID: "d0cb9dd6-b140-4616-8905-21f89f14a61c"). InnerVolumeSpecName "kube-api-access-kr8fq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:55:55.342977 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:55.342941    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0cb9dd6-b140-4616-8905-21f89f14a61c-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:55:55.342977 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:55.342969    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0cb9dd6-b140-4616-8905-21f89f14a61c-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:55:55.342977 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:55.342979    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kr8fq\" (UniqueName: \"kubernetes.io/projected/d0cb9dd6-b140-4616-8905-21f89f14a61c-kube-api-access-kr8fq\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:55:56.009972 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:56.009932    2549 generic.go:356] "Generic (PLEG): container finished" podID="d0cb9dd6-b140-4616-8905-21f89f14a61c" containerID="3a9a369a035832bf72d00f74c62d3d80f557a8c39242594e8a1233652d7aaf0c" exitCode=0
Sep 10 17:55:56.010392 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:56.009975    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"
Sep 10 17:55:56.010392 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:56.010023    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62" event={"ID":"d0cb9dd6-b140-4616-8905-21f89f14a61c","Type":"ContainerDied","Data":"3a9a369a035832bf72d00f74c62d3d80f557a8c39242594e8a1233652d7aaf0c"}
Sep 10 17:55:56.010392 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:56.010073    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62" event={"ID":"d0cb9dd6-b140-4616-8905-21f89f14a61c","Type":"ContainerDied","Data":"75f489be0f596ca9f2708887e20ba88beb2524151a0ca892f1af0b6e70607b84"}
Sep 10 17:55:56.010392 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:56.010097    2549 scope.go:117] "RemoveContainer" containerID="3a9a369a035832bf72d00f74c62d3d80f557a8c39242594e8a1233652d7aaf0c"
Sep 10 17:55:56.019409 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:56.019385    2549 scope.go:117] "RemoveContainer" containerID="3a9a369a035832bf72d00f74c62d3d80f557a8c39242594e8a1233652d7aaf0c"
Sep 10 17:55:56.019728 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:55:56.019686    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a9a369a035832bf72d00f74c62d3d80f557a8c39242594e8a1233652d7aaf0c\": container with ID starting with 3a9a369a035832bf72d00f74c62d3d80f557a8c39242594e8a1233652d7aaf0c not found: ID does not exist" containerID="3a9a369a035832bf72d00f74c62d3d80f557a8c39242594e8a1233652d7aaf0c"
Sep 10 17:55:56.019812 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:56.019744    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a9a369a035832bf72d00f74c62d3d80f557a8c39242594e8a1233652d7aaf0c"} err="failed to get container status \"3a9a369a035832bf72d00f74c62d3d80f557a8c39242594e8a1233652d7aaf0c\": rpc error: code = NotFound desc = could not find container \"3a9a369a035832bf72d00f74c62d3d80f557a8c39242594e8a1233652d7aaf0c\": container with ID starting with 3a9a369a035832bf72d00f74c62d3d80f557a8c39242594e8a1233652d7aaf0c not found: ID does not exist"
Sep 10 17:55:56.035766 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:56.035736    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"]
Sep 10 17:55:56.039646 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:56.039613    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-826cc2b1-76c88f6658-pvw62"]
Sep 10 17:55:58.034786 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:58.034750    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0cb9dd6-b140-4616-8905-21f89f14a61c" path="/var/lib/kubelet/pods/d0cb9dd6-b140-4616-8905-21f89f14a61c/volumes"
Sep 10 17:55:59.441624 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.441591    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"]
Sep 10 17:55:59.442016 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.441965    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0cb9dd6-b140-4616-8905-21f89f14a61c" containerName="maas-api"
Sep 10 17:55:59.442016 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.441977    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0cb9dd6-b140-4616-8905-21f89f14a61c" containerName="maas-api"
Sep 10 17:55:59.442102 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.442044    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0cb9dd6-b140-4616-8905-21f89f14a61c" containerName="maas-api"
Sep 10 17:55:59.446319 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.446296    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:55:59.448912 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.448884    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9584e914-serving-cert\""
Sep 10 17:55:59.456752 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.456726    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"]
Sep 10 17:55:59.477718 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.477663    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrrp8\" (UniqueName: \"kubernetes.io/projected/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-kube-api-access-mrrp8\") pod \"maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:55:59.477873 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.477801    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-metrics-certs\") pod \"maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:55:59.477916 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.477873    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-openshift-service-ca\") pod \"maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:55:59.477953 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.477916    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-maas-api-tls\") pod \"maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:55:59.579022 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.578982    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrrp8\" (UniqueName: \"kubernetes.io/projected/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-kube-api-access-mrrp8\") pod \"maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:55:59.579198 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.579143    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-metrics-certs\") pod \"maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:55:59.579265 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.579246    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-openshift-service-ca\") pod \"maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:55:59.579335 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.579309    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-maas-api-tls\") pod \"maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:55:59.579466 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:55:59.579444    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-serving-cert: secret "maas-api-e2e-ait-9584e914-serving-cert" not found
Sep 10 17:55:59.579576 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:55:59.579556    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-maas-api-tls podName:935e2f1a-dc90-4d44-9ea8-fc47d9533a9c nodeName:}" failed. No retries permitted until 2026-09-10 17:56:00.079531509 +0000 UTC m=+884.570583309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-maas-api-tls") pod "maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx" (UID: "935e2f1a-dc90-4d44-9ea8-fc47d9533a9c") : secret "maas-api-e2e-ait-9584e914-serving-cert" not found
Sep 10 17:55:59.580003 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.579985    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-openshift-service-ca\") pod \"maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:55:59.581484 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.581458    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-metrics-certs\") pod \"maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:55:59.592073 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:55:59.592047    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrrp8\" (UniqueName: \"kubernetes.io/projected/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-kube-api-access-mrrp8\") pod \"maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:56:00.084519 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:00.084465    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-maas-api-tls\") pod \"maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:56:00.086869 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:00.086841    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-maas-api-tls\") pod \"maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:56:00.357913 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:00.357820    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:56:00.635074 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:00.635045    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"]
Sep 10 17:56:00.637604 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:56:00.637572    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod935e2f1a_dc90_4d44_9ea8_fc47d9533a9c.slice/crio-fdd85fc955e086d5d12ff84005a9451ca6b1d56100b8c7439264db7b59e55cb3 WatchSource:0}: Error finding container fdd85fc955e086d5d12ff84005a9451ca6b1d56100b8c7439264db7b59e55cb3: Status 404 returned error can't find the container with id fdd85fc955e086d5d12ff84005a9451ca6b1d56100b8c7439264db7b59e55cb3
Sep 10 17:56:01.035858 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:01.035821    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx" event={"ID":"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c","Type":"ContainerStarted","Data":"fdd85fc955e086d5d12ff84005a9451ca6b1d56100b8c7439264db7b59e55cb3"}
Sep 10 17:56:02.041765 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:02.041726    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx" event={"ID":"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c","Type":"ContainerStarted","Data":"d1f0555a0fbf1ec9d5b0af92fd87a2c19bbd7aa3c053208b541d95e01f9b9b2b"}
Sep 10 17:56:02.042153 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:02.041839    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:56:02.062264 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:02.062203    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx" podStartSLOduration=2.606115012 podStartE2EDuration="3.06218623s" totalImagesPullingTime="456.071218ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:55:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:56:02.061199652 +0000 UTC m=+886.552251460" watchObservedRunningTime="2026-09-10 17:56:02.06218623 +0000 UTC m=+886.553238035"
Sep 10 17:56:02.499397 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:02.499359    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:56:08.051901 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:08.051824    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:56:10.783810 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.783776    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"]
Sep 10 17:56:10.787735 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.787712    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.790657 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.790631    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-bcdd7c\"/\"kube-root-ca.crt\""
Sep 10 17:56:10.791797 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.791772    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-bcdd7c\"/\"openshift-service-ca.crt\""
Sep 10 17:56:10.791906 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.791889    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-bcdd7c\"/\"default-dockercfg-7h8gd\""
Sep 10 17:56:10.791951 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.791910    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-bcdd7c\"/\"autores-model-bcdd7c-kserve-self-signed-certs\""
Sep 10 17:56:10.799610 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.799586    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"]
Sep 10 17:56:10.879268 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.879218    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-dshm\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.879268 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.879287    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-model-cache\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.879538 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.879353    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-home\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.879538 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.879389    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-tmp-dir\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.879538 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.879497    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-tls-certs\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.879688 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.879544    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9bhb\" (UniqueName: \"kubernetes.io/projected/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-kube-api-access-v9bhb\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.980863 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.980809    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-dshm\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.981039 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.980877    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-model-cache\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.981039 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.980921    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-home\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.981039 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.980948    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-tmp-dir\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.981039 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.981027    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-tls-certs\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.981264 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.981067    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9bhb\" (UniqueName: \"kubernetes.io/projected/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-kube-api-access-v9bhb\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.981381 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.981351    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-model-cache\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.981438 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.981367    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-home\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.981438 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.981417    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-tmp-dir\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.983199 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.983170    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-dshm\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.983541 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.983521    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-tls-certs\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:10.989433 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:10.989408    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9bhb\" (UniqueName: \"kubernetes.io/projected/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-kube-api-access-v9bhb\") pod \"autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") " pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:11.097616 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:11.097527    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:11.369530 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:11.369503    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"]
Sep 10 17:56:11.371420 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:56:11.371378    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod527b2b9f_fd70_4145_8b08_82fbaa63e6c2.slice/crio-53213d85e9ff02475b35be7663ea6e76790375bfa81663f5d63d6c3360c6d82a WatchSource:0}: Error finding container 53213d85e9ff02475b35be7663ea6e76790375bfa81663f5d63d6c3360c6d82a: Status 404 returned error can't find the container with id 53213d85e9ff02475b35be7663ea6e76790375bfa81663f5d63d6c3360c6d82a
Sep 10 17:56:12.099184 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:12.099143    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs" event={"ID":"527b2b9f-fd70-4145-8b08-82fbaa63e6c2","Type":"ContainerStarted","Data":"eb79d2489c2b6ead7170b2b8b37a60194b459eac6bba10a6da7ede78a4110b1b"}
Sep 10 17:56:12.099184 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:12.099192    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs" event={"ID":"527b2b9f-fd70-4145-8b08-82fbaa63e6c2","Type":"ContainerStarted","Data":"53213d85e9ff02475b35be7663ea6e76790375bfa81663f5d63d6c3360c6d82a"}
Sep 10 17:56:12.117426 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:12.117357    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs" podStartSLOduration=2.11733678 podStartE2EDuration="2.11733678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:56:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:56:12.116836819 +0000 UTC m=+896.607888686" watchObservedRunningTime="2026-09-10 17:56:12.11733678 +0000 UTC m=+896.608388589"
Sep 10 17:56:20.007416 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:20.007370    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:56:21.097896 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:21.097843    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:21.098397 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:21.097914    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:21.110402 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:21.110372    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:21.156975 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:21.156944    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:56:29.919219 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:29.919184    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"]
Sep 10 17:56:29.919667 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:29.919434    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx" podUID="935e2f1a-dc90-4d44-9ea8-fc47d9533a9c" containerName="maas-api" containerID="cri-o://d1f0555a0fbf1ec9d5b0af92fd87a2c19bbd7aa3c053208b541d95e01f9b9b2b" gracePeriod=30
Sep 10 17:56:30.172101 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.172041    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:56:30.188078 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.188046    2549 generic.go:356] "Generic (PLEG): container finished" podID="935e2f1a-dc90-4d44-9ea8-fc47d9533a9c" containerID="d1f0555a0fbf1ec9d5b0af92fd87a2c19bbd7aa3c053208b541d95e01f9b9b2b" exitCode=0
Sep 10 17:56:30.188248 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.188089    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"
Sep 10 17:56:30.188248 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.188126    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx" event={"ID":"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c","Type":"ContainerDied","Data":"d1f0555a0fbf1ec9d5b0af92fd87a2c19bbd7aa3c053208b541d95e01f9b9b2b"}
Sep 10 17:56:30.188248 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.188180    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx" event={"ID":"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c","Type":"ContainerDied","Data":"fdd85fc955e086d5d12ff84005a9451ca6b1d56100b8c7439264db7b59e55cb3"}
Sep 10 17:56:30.188248 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.188204    2549 scope.go:117] "RemoveContainer" containerID="d1f0555a0fbf1ec9d5b0af92fd87a2c19bbd7aa3c053208b541d95e01f9b9b2b"
Sep 10 17:56:30.199347 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.199324    2549 scope.go:117] "RemoveContainer" containerID="d1f0555a0fbf1ec9d5b0af92fd87a2c19bbd7aa3c053208b541d95e01f9b9b2b"
Sep 10 17:56:30.199712 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:56:30.199671    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1f0555a0fbf1ec9d5b0af92fd87a2c19bbd7aa3c053208b541d95e01f9b9b2b\": container with ID starting with d1f0555a0fbf1ec9d5b0af92fd87a2c19bbd7aa3c053208b541d95e01f9b9b2b not found: ID does not exist" containerID="d1f0555a0fbf1ec9d5b0af92fd87a2c19bbd7aa3c053208b541d95e01f9b9b2b"
Sep 10 17:56:30.199839 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.199797    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1f0555a0fbf1ec9d5b0af92fd87a2c19bbd7aa3c053208b541d95e01f9b9b2b"} err="failed to get container status \"d1f0555a0fbf1ec9d5b0af92fd87a2c19bbd7aa3c053208b541d95e01f9b9b2b\": rpc error: code = NotFound desc = could not find container \"d1f0555a0fbf1ec9d5b0af92fd87a2c19bbd7aa3c053208b541d95e01f9b9b2b\": container with ID starting with d1f0555a0fbf1ec9d5b0af92fd87a2c19bbd7aa3c053208b541d95e01f9b9b2b not found: ID does not exist"
Sep 10 17:56:30.263209 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.263180    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-metrics-certs\") pod \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") "
Sep 10 17:56:30.263366 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.263217    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-maas-api-tls\") pod \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") "
Sep 10 17:56:30.263366 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.263245    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrrp8\" (UniqueName: \"kubernetes.io/projected/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-kube-api-access-mrrp8\") pod \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") "
Sep 10 17:56:30.263366 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.263329    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-openshift-service-ca\") pod \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\" (UID: \"935e2f1a-dc90-4d44-9ea8-fc47d9533a9c\") "
Sep 10 17:56:30.263853 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.263822    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "935e2f1a-dc90-4d44-9ea8-fc47d9533a9c" (UID: "935e2f1a-dc90-4d44-9ea8-fc47d9533a9c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:56:30.265342 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.265311    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-kube-api-access-mrrp8" (OuterVolumeSpecName: "kube-api-access-mrrp8") pod "935e2f1a-dc90-4d44-9ea8-fc47d9533a9c" (UID: "935e2f1a-dc90-4d44-9ea8-fc47d9533a9c"). InnerVolumeSpecName "kube-api-access-mrrp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:56:30.265553 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.265524    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "935e2f1a-dc90-4d44-9ea8-fc47d9533a9c" (UID: "935e2f1a-dc90-4d44-9ea8-fc47d9533a9c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:56:30.265857 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.265832    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "935e2f1a-dc90-4d44-9ea8-fc47d9533a9c" (UID: "935e2f1a-dc90-4d44-9ea8-fc47d9533a9c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:56:30.364998 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.364962    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:56:30.364998 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.364991    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:56:30.364998 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.365002    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:56:30.364998 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.365010    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrrp8\" (UniqueName: \"kubernetes.io/projected/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c-kube-api-access-mrrp8\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:56:30.510661 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.510622    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"]
Sep 10 17:56:30.512275 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:30.512250    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9584e914-9c8c747c9-8lxrx"]
Sep 10 17:56:32.029852 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:32.029817    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="935e2f1a-dc90-4d44-9ea8-fc47d9533a9c" path="/var/lib/kubelet/pods/935e2f1a-dc90-4d44-9ea8-fc47d9533a9c/volumes"
Sep 10 17:56:34.497226 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.497187    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"]
Sep 10 17:56:34.499643 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.497564    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="935e2f1a-dc90-4d44-9ea8-fc47d9533a9c" containerName="maas-api"
Sep 10 17:56:34.499643 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.497575    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="935e2f1a-dc90-4d44-9ea8-fc47d9533a9c" containerName="maas-api"
Sep 10 17:56:34.499643 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.497640    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="935e2f1a-dc90-4d44-9ea8-fc47d9533a9c" containerName="maas-api"
Sep 10 17:56:34.500482 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.500465    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.504728 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.504688    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c34c13\"/\"kube-root-ca.crt\""
Sep 10 17:56:34.504873 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.504726    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c34c13\"/\"openshift-service-ca.crt\""
Sep 10 17:56:34.504873 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.504745    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c34c13\"/\"default-dockercfg-nhd4d\""
Sep 10 17:56:34.504873 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.504746    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c34c13\"/\"test-model-c34c13-kserve-self-signed-certs\""
Sep 10 17:56:34.516666 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.516637    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"]
Sep 10 17:56:34.603968 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.603925    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-home\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.603968 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.603973    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-dshm\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.604173 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.604018    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46bc48fe-eb5f-44cd-8aca-5a130273f61f-tls-certs\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.604173 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.604082    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-model-cache\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.604173 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.604097    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kncgm\" (UniqueName: \"kubernetes.io/projected/46bc48fe-eb5f-44cd-8aca-5a130273f61f-kube-api-access-kncgm\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.604173 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.604116    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-tmp-dir\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.704878 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.704822    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-model-cache\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.704878 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.704876    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kncgm\" (UniqueName: \"kubernetes.io/projected/46bc48fe-eb5f-44cd-8aca-5a130273f61f-kube-api-access-kncgm\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.705144 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.704904    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-tmp-dir\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.705144 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.704936    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-home\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.705144 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.704963    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-dshm\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.705144 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.704985    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46bc48fe-eb5f-44cd-8aca-5a130273f61f-tls-certs\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.705363 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.705324    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-model-cache\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.705421 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.705356    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-tmp-dir\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.705421 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.705363    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-home\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.707242 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.707222    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-dshm\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.707488 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.707471    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46bc48fe-eb5f-44cd-8aca-5a130273f61f-tls-certs\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.716915 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.716849    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kncgm\" (UniqueName: \"kubernetes.io/projected/46bc48fe-eb5f-44cd-8aca-5a130273f61f-kube-api-access-kncgm\") pod \"test-model-c34c13-kserve-54cdd49d87-9hdzz\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") " pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:34.809063 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:34.808958    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:35.083395 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.083302    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"]
Sep 10 17:56:35.091492 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:56:35.091458    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46bc48fe_eb5f_44cd_8aca_5a130273f61f.slice/crio-a6ae94c0a1f6ddd319805a79127670e9294f48b45f648f89f9ff8f416e6130c1 WatchSource:0}: Error finding container a6ae94c0a1f6ddd319805a79127670e9294f48b45f648f89f9ff8f416e6130c1: Status 404 returned error can't find the container with id a6ae94c0a1f6ddd319805a79127670e9294f48b45f648f89f9ff8f416e6130c1
Sep 10 17:56:35.213828 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.213792    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz" event={"ID":"46bc48fe-eb5f-44cd-8aca-5a130273f61f","Type":"ContainerStarted","Data":"b3263e1997eac7c1fc5fc3433d937bab39a211cb6cdff3602cc818b79071f179"}
Sep 10 17:56:35.213828 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.213833    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz" event={"ID":"46bc48fe-eb5f-44cd-8aca-5a130273f61f","Type":"ContainerStarted","Data":"a6ae94c0a1f6ddd319805a79127670e9294f48b45f648f89f9ff8f416e6130c1"}
Sep 10 17:56:35.235199 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.235140    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz" podStartSLOduration=1.235123508 podStartE2EDuration="1.235123508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:56:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:56:35.234550478 +0000 UTC m=+919.725602287" watchObservedRunningTime="2026-09-10 17:56:35.235123508 +0000 UTC m=+919.726175720"
Sep 10 17:56:35.714265 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.714222    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"]
Sep 10 17:56:35.718630 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.718604    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:35.721680 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.721653    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-d0df615c-serving-cert\""
Sep 10 17:56:35.730428 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.730402    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"]
Sep 10 17:56:35.816708 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.816660    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/878b5ac3-6c7a-410c-9588-71e33ced3bd8-maas-api-tls\") pod \"maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:35.816897 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.816748    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldmbx\" (UniqueName: \"kubernetes.io/projected/878b5ac3-6c7a-410c-9588-71e33ced3bd8-kube-api-access-ldmbx\") pod \"maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:35.816897 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.816783    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/878b5ac3-6c7a-410c-9588-71e33ced3bd8-openshift-service-ca\") pod \"maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:35.816897 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.816808    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/878b5ac3-6c7a-410c-9588-71e33ced3bd8-metrics-certs\") pod \"maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:35.917851 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.917812    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/878b5ac3-6c7a-410c-9588-71e33ced3bd8-maas-api-tls\") pod \"maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:35.918028 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.917874    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldmbx\" (UniqueName: \"kubernetes.io/projected/878b5ac3-6c7a-410c-9588-71e33ced3bd8-kube-api-access-ldmbx\") pod \"maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:35.918028 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.917914    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/878b5ac3-6c7a-410c-9588-71e33ced3bd8-openshift-service-ca\") pod \"maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:35.918028 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.917939    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/878b5ac3-6c7a-410c-9588-71e33ced3bd8-metrics-certs\") pod \"maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:35.918028 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:56:35.917986    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-serving-cert: secret "maas-api-e2e-derive-d0df615c-serving-cert" not found
Sep 10 17:56:35.918212 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:56:35.918071    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/878b5ac3-6c7a-410c-9588-71e33ced3bd8-maas-api-tls podName:878b5ac3-6c7a-410c-9588-71e33ced3bd8 nodeName:}" failed. No retries permitted until 2026-09-10 17:56:36.418049422 +0000 UTC m=+920.909101222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/878b5ac3-6c7a-410c-9588-71e33ced3bd8-maas-api-tls") pod "maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv" (UID: "878b5ac3-6c7a-410c-9588-71e33ced3bd8") : secret "maas-api-e2e-derive-d0df615c-serving-cert" not found
Sep 10 17:56:35.918708 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.918665    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/878b5ac3-6c7a-410c-9588-71e33ced3bd8-openshift-service-ca\") pod \"maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:35.920527 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.920506    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/878b5ac3-6c7a-410c-9588-71e33ced3bd8-metrics-certs\") pod \"maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:35.929183 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:35.929157    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldmbx\" (UniqueName: \"kubernetes.io/projected/878b5ac3-6c7a-410c-9588-71e33ced3bd8-kube-api-access-ldmbx\") pod \"maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:36.421870 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:36.421828    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/878b5ac3-6c7a-410c-9588-71e33ced3bd8-maas-api-tls\") pod \"maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:36.424536 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:36.424502    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/878b5ac3-6c7a-410c-9588-71e33ced3bd8-maas-api-tls\") pod \"maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:36.632455 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:36.632415    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:36.907739 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:36.907703    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"]
Sep 10 17:56:36.911267 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:56:36.911238    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod878b5ac3_6c7a_410c_9588_71e33ced3bd8.slice/crio-ba4ee3bb883fbadc9705d8ad9a04becb0e49848d4451021119131ccfb01afb05 WatchSource:0}: Error finding container ba4ee3bb883fbadc9705d8ad9a04becb0e49848d4451021119131ccfb01afb05: Status 404 returned error can't find the container with id ba4ee3bb883fbadc9705d8ad9a04becb0e49848d4451021119131ccfb01afb05
Sep 10 17:56:37.224193 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:37.224148    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv" event={"ID":"878b5ac3-6c7a-410c-9588-71e33ced3bd8","Type":"ContainerStarted","Data":"ba4ee3bb883fbadc9705d8ad9a04becb0e49848d4451021119131ccfb01afb05"}
Sep 10 17:56:38.230392 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:38.230348    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv" event={"ID":"878b5ac3-6c7a-410c-9588-71e33ced3bd8","Type":"ContainerStarted","Data":"bf719317f6721c790e2b52fe684b13c6b4219a93bfee64a5e4e3c2815ee2168c"}
Sep 10 17:56:38.230797 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:38.230450    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:38.249570 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:38.249516    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv" podStartSLOduration=2.898119974 podStartE2EDuration="3.249498371s" totalImagesPullingTime="351.378397ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:56:38.248735784 +0000 UTC m=+922.739787592" watchObservedRunningTime="2026-09-10 17:56:38.249498371 +0000 UTC m=+922.740550161"
Sep 10 17:56:40.805473 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:40.805429    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:56:44.240983 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:44.240953    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:56:44.779094 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:44.779059    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"]
Sep 10 17:56:44.809585 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:44.809557    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:44.809757 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:44.809606    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:44.821981 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:44.821958    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:45.274436 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:45.274407    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:56:55.574123 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:55.574091    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"]
Sep 10 17:56:55.574524 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:55.574311    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz" podUID="62539a9b-5ede-4aab-91f8-4dd7242598ac" containerName="maas-api" containerID="cri-o://5ce5de4f63927057dc8fd7aba952f0d4ab545fc5869eecf114664a831a565992" gracePeriod=30
Sep 10 17:56:55.821347 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:55.821321    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:56:55.897492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:55.897402    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62539a9b-5ede-4aab-91f8-4dd7242598ac-openshift-service-ca\") pod \"62539a9b-5ede-4aab-91f8-4dd7242598ac\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") "
Sep 10 17:56:55.897492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:55.897454    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhw4m\" (UniqueName: \"kubernetes.io/projected/62539a9b-5ede-4aab-91f8-4dd7242598ac-kube-api-access-qhw4m\") pod \"62539a9b-5ede-4aab-91f8-4dd7242598ac\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") "
Sep 10 17:56:55.897735 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:55.897503    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62539a9b-5ede-4aab-91f8-4dd7242598ac-maas-api-tls\") pod \"62539a9b-5ede-4aab-91f8-4dd7242598ac\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") "
Sep 10 17:56:55.897735 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:55.897525    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62539a9b-5ede-4aab-91f8-4dd7242598ac-metrics-certs\") pod \"62539a9b-5ede-4aab-91f8-4dd7242598ac\" (UID: \"62539a9b-5ede-4aab-91f8-4dd7242598ac\") "
Sep 10 17:56:55.897836 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:55.897784    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62539a9b-5ede-4aab-91f8-4dd7242598ac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "62539a9b-5ede-4aab-91f8-4dd7242598ac" (UID: "62539a9b-5ede-4aab-91f8-4dd7242598ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:56:55.897892 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:55.897856    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62539a9b-5ede-4aab-91f8-4dd7242598ac-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:56:55.899652 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:55.899622    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62539a9b-5ede-4aab-91f8-4dd7242598ac-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "62539a9b-5ede-4aab-91f8-4dd7242598ac" (UID: "62539a9b-5ede-4aab-91f8-4dd7242598ac"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:56:55.899787 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:55.899670    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62539a9b-5ede-4aab-91f8-4dd7242598ac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "62539a9b-5ede-4aab-91f8-4dd7242598ac" (UID: "62539a9b-5ede-4aab-91f8-4dd7242598ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:56:55.899787 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:55.899768    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62539a9b-5ede-4aab-91f8-4dd7242598ac-kube-api-access-qhw4m" (OuterVolumeSpecName: "kube-api-access-qhw4m") pod "62539a9b-5ede-4aab-91f8-4dd7242598ac" (UID: "62539a9b-5ede-4aab-91f8-4dd7242598ac"). InnerVolumeSpecName "kube-api-access-qhw4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:56:55.999063 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:55.999026    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62539a9b-5ede-4aab-91f8-4dd7242598ac-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:56:55.999063 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:55.999056    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62539a9b-5ede-4aab-91f8-4dd7242598ac-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:56:55.999063 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:55.999066    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhw4m\" (UniqueName: \"kubernetes.io/projected/62539a9b-5ede-4aab-91f8-4dd7242598ac-kube-api-access-qhw4m\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:56:56.314576 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:56.314537    2549 generic.go:356] "Generic (PLEG): container finished" podID="62539a9b-5ede-4aab-91f8-4dd7242598ac" containerID="5ce5de4f63927057dc8fd7aba952f0d4ab545fc5869eecf114664a831a565992" exitCode=0
Sep 10 17:56:56.314783 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:56.314585    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"
Sep 10 17:56:56.314783 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:56.314579    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz" event={"ID":"62539a9b-5ede-4aab-91f8-4dd7242598ac","Type":"ContainerDied","Data":"5ce5de4f63927057dc8fd7aba952f0d4ab545fc5869eecf114664a831a565992"}
Sep 10 17:56:56.314783 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:56.314755    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz" event={"ID":"62539a9b-5ede-4aab-91f8-4dd7242598ac","Type":"ContainerDied","Data":"880e923351a54f96924bd5d06811cc55a76c36c0ae0118c1978d42a309459b82"}
Sep 10 17:56:56.314937 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:56.314790    2549 scope.go:117] "RemoveContainer" containerID="5ce5de4f63927057dc8fd7aba952f0d4ab545fc5869eecf114664a831a565992"
Sep 10 17:56:56.326421 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:56.326402    2549 scope.go:117] "RemoveContainer" containerID="5ce5de4f63927057dc8fd7aba952f0d4ab545fc5869eecf114664a831a565992"
Sep 10 17:56:56.326667 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:56:56.326647    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ce5de4f63927057dc8fd7aba952f0d4ab545fc5869eecf114664a831a565992\": container with ID starting with 5ce5de4f63927057dc8fd7aba952f0d4ab545fc5869eecf114664a831a565992 not found: ID does not exist" containerID="5ce5de4f63927057dc8fd7aba952f0d4ab545fc5869eecf114664a831a565992"
Sep 10 17:56:56.326743 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:56.326674    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ce5de4f63927057dc8fd7aba952f0d4ab545fc5869eecf114664a831a565992"} err="failed to get container status \"5ce5de4f63927057dc8fd7aba952f0d4ab545fc5869eecf114664a831a565992\": rpc error: code = NotFound desc = could not find container \"5ce5de4f63927057dc8fd7aba952f0d4ab545fc5869eecf114664a831a565992\": container with ID starting with 5ce5de4f63927057dc8fd7aba952f0d4ab545fc5869eecf114664a831a565992 not found: ID does not exist"
Sep 10 17:56:56.334537 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:56.334514    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"]
Sep 10 17:56:56.342308 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:56.342284    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-bcdd7c-565b87f5f8-tdbbz"]
Sep 10 17:56:58.029460 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:58.029426    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62539a9b-5ede-4aab-91f8-4dd7242598ac" path="/var/lib/kubelet/pods/62539a9b-5ede-4aab-91f8-4dd7242598ac/volumes"
Sep 10 17:56:59.799257 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:56:59.799208    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs" podUID="527b2b9f-fd70-4145-8b08-82fbaa63e6c2" containerName="main" containerID="cri-o://eb79d2489c2b6ead7170b2b8b37a60194b459eac6bba10a6da7ede78a4110b1b" gracePeriod=45
Sep 10 17:57:00.056538 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.056480    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:57:00.140031 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.139996    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-home\") pod \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") "
Sep 10 17:57:00.140230 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.140054    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-tls-certs\") pod \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") "
Sep 10 17:57:00.140230 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.140114    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-dshm\") pod \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") "
Sep 10 17:57:00.140230 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.140146    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-model-cache\") pod \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") "
Sep 10 17:57:00.140230 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.140184    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9bhb\" (UniqueName: \"kubernetes.io/projected/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-kube-api-access-v9bhb\") pod \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") "
Sep 10 17:57:00.140230 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.140208    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-tmp-dir\") pod \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\" (UID: \"527b2b9f-fd70-4145-8b08-82fbaa63e6c2\") "
Sep 10 17:57:00.140482 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.140313    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-home" (OuterVolumeSpecName: "home") pod "527b2b9f-fd70-4145-8b08-82fbaa63e6c2" (UID: "527b2b9f-fd70-4145-8b08-82fbaa63e6c2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:57:00.140482 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.140468    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-model-cache" (OuterVolumeSpecName: "model-cache") pod "527b2b9f-fd70-4145-8b08-82fbaa63e6c2" (UID: "527b2b9f-fd70-4145-8b08-82fbaa63e6c2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:57:00.140593 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.140531    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-home\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:57:00.140593 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.140543    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-model-cache\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:57:00.140702 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.140649    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "527b2b9f-fd70-4145-8b08-82fbaa63e6c2" (UID: "527b2b9f-fd70-4145-8b08-82fbaa63e6c2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:57:00.142471 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.142437    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-kube-api-access-v9bhb" (OuterVolumeSpecName: "kube-api-access-v9bhb") pod "527b2b9f-fd70-4145-8b08-82fbaa63e6c2" (UID: "527b2b9f-fd70-4145-8b08-82fbaa63e6c2"). InnerVolumeSpecName "kube-api-access-v9bhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:57:00.142649 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.142634    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-dshm" (OuterVolumeSpecName: "dshm") pod "527b2b9f-fd70-4145-8b08-82fbaa63e6c2" (UID: "527b2b9f-fd70-4145-8b08-82fbaa63e6c2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:57:00.142751 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.142713    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "527b2b9f-fd70-4145-8b08-82fbaa63e6c2" (UID: "527b2b9f-fd70-4145-8b08-82fbaa63e6c2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:57:00.241381 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.241345    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9bhb\" (UniqueName: \"kubernetes.io/projected/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-kube-api-access-v9bhb\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:57:00.241381 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.241380    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-tmp-dir\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:57:00.241582 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.241394    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-tls-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:57:00.241582 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.241405    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/527b2b9f-fd70-4145-8b08-82fbaa63e6c2-dshm\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:57:00.335586 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.335501    2549 generic.go:356] "Generic (PLEG): container finished" podID="527b2b9f-fd70-4145-8b08-82fbaa63e6c2" containerID="eb79d2489c2b6ead7170b2b8b37a60194b459eac6bba10a6da7ede78a4110b1b" exitCode=0
Sep 10 17:57:00.335586 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.335542    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"
Sep 10 17:57:00.335803 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.335580    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs" event={"ID":"527b2b9f-fd70-4145-8b08-82fbaa63e6c2","Type":"ContainerDied","Data":"eb79d2489c2b6ead7170b2b8b37a60194b459eac6bba10a6da7ede78a4110b1b"}
Sep 10 17:57:00.335803 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.335623    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs" event={"ID":"527b2b9f-fd70-4145-8b08-82fbaa63e6c2","Type":"ContainerDied","Data":"53213d85e9ff02475b35be7663ea6e76790375bfa81663f5d63d6c3360c6d82a"}
Sep 10 17:57:00.335803 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.335641    2549 scope.go:117] "RemoveContainer" containerID="eb79d2489c2b6ead7170b2b8b37a60194b459eac6bba10a6da7ede78a4110b1b"
Sep 10 17:57:00.345038 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.345020    2549 scope.go:117] "RemoveContainer" containerID="eb79d2489c2b6ead7170b2b8b37a60194b459eac6bba10a6da7ede78a4110b1b"
Sep 10 17:57:00.345318 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:57:00.345297    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb79d2489c2b6ead7170b2b8b37a60194b459eac6bba10a6da7ede78a4110b1b\": container with ID starting with eb79d2489c2b6ead7170b2b8b37a60194b459eac6bba10a6da7ede78a4110b1b not found: ID does not exist" containerID="eb79d2489c2b6ead7170b2b8b37a60194b459eac6bba10a6da7ede78a4110b1b"
Sep 10 17:57:00.345376 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.345330    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb79d2489c2b6ead7170b2b8b37a60194b459eac6bba10a6da7ede78a4110b1b"} err="failed to get container status \"eb79d2489c2b6ead7170b2b8b37a60194b459eac6bba10a6da7ede78a4110b1b\": rpc error: code = NotFound desc = could not find container \"eb79d2489c2b6ead7170b2b8b37a60194b459eac6bba10a6da7ede78a4110b1b\": container with ID starting with eb79d2489c2b6ead7170b2b8b37a60194b459eac6bba10a6da7ede78a4110b1b not found: ID does not exist"
Sep 10 17:57:00.358565 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.358534    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"]
Sep 10 17:57:00.365277 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.365253    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-bcdd7c/autores-model-bcdd7c-kserve-5d77d97dcf-5z8hs"]
Sep 10 17:57:00.702985 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:00.702949    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:57:02.029789 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:02.029754    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="527b2b9f-fd70-4145-8b08-82fbaa63e6c2" path="/var/lib/kubelet/pods/527b2b9f-fd70-4145-8b08-82fbaa63e6c2/volumes"
Sep 10 17:57:06.414415 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:06.414367    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"]
Sep 10 17:57:06.414884 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:06.414606    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv" podUID="878b5ac3-6c7a-410c-9588-71e33ced3bd8" containerName="maas-api" containerID="cri-o://bf719317f6721c790e2b52fe684b13c6b4219a93bfee64a5e4e3c2815ee2168c" gracePeriod=30
Sep 10 17:57:06.684495 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:06.684469    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:57:06.805743 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:06.805676    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/878b5ac3-6c7a-410c-9588-71e33ced3bd8-metrics-certs\") pod \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") "
Sep 10 17:57:06.805931 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:06.805775    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/878b5ac3-6c7a-410c-9588-71e33ced3bd8-openshift-service-ca\") pod \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") "
Sep 10 17:57:06.805931 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:06.805867    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/878b5ac3-6c7a-410c-9588-71e33ced3bd8-maas-api-tls\") pod \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") "
Sep 10 17:57:06.805931 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:06.805931    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldmbx\" (UniqueName: \"kubernetes.io/projected/878b5ac3-6c7a-410c-9588-71e33ced3bd8-kube-api-access-ldmbx\") pod \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\" (UID: \"878b5ac3-6c7a-410c-9588-71e33ced3bd8\") "
Sep 10 17:57:06.806172 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:06.806139    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/878b5ac3-6c7a-410c-9588-71e33ced3bd8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "878b5ac3-6c7a-410c-9588-71e33ced3bd8" (UID: "878b5ac3-6c7a-410c-9588-71e33ced3bd8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:57:06.806257 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:06.806243    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/878b5ac3-6c7a-410c-9588-71e33ced3bd8-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:57:06.807991 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:06.807962    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/878b5ac3-6c7a-410c-9588-71e33ced3bd8-kube-api-access-ldmbx" (OuterVolumeSpecName: "kube-api-access-ldmbx") pod "878b5ac3-6c7a-410c-9588-71e33ced3bd8" (UID: "878b5ac3-6c7a-410c-9588-71e33ced3bd8"). InnerVolumeSpecName "kube-api-access-ldmbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:57:06.808112 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:06.807966    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/878b5ac3-6c7a-410c-9588-71e33ced3bd8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "878b5ac3-6c7a-410c-9588-71e33ced3bd8" (UID: "878b5ac3-6c7a-410c-9588-71e33ced3bd8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:57:06.808112 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:06.807994    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/878b5ac3-6c7a-410c-9588-71e33ced3bd8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "878b5ac3-6c7a-410c-9588-71e33ced3bd8" (UID: "878b5ac3-6c7a-410c-9588-71e33ced3bd8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:57:06.907619 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:06.907587    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/878b5ac3-6c7a-410c-9588-71e33ced3bd8-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:57:06.907619 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:06.907613    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/878b5ac3-6c7a-410c-9588-71e33ced3bd8-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:57:06.907619 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:06.907622    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldmbx\" (UniqueName: \"kubernetes.io/projected/878b5ac3-6c7a-410c-9588-71e33ced3bd8-kube-api-access-ldmbx\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:57:07.369398 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.369363    2549 generic.go:356] "Generic (PLEG): container finished" podID="878b5ac3-6c7a-410c-9588-71e33ced3bd8" containerID="bf719317f6721c790e2b52fe684b13c6b4219a93bfee64a5e4e3c2815ee2168c" exitCode=0
Sep 10 17:57:07.369563 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.369412    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"
Sep 10 17:57:07.369563 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.369417    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv" event={"ID":"878b5ac3-6c7a-410c-9588-71e33ced3bd8","Type":"ContainerDied","Data":"bf719317f6721c790e2b52fe684b13c6b4219a93bfee64a5e4e3c2815ee2168c"}
Sep 10 17:57:07.369563 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.369515    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv" event={"ID":"878b5ac3-6c7a-410c-9588-71e33ced3bd8","Type":"ContainerDied","Data":"ba4ee3bb883fbadc9705d8ad9a04becb0e49848d4451021119131ccfb01afb05"}
Sep 10 17:57:07.369563 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.369537    2549 scope.go:117] "RemoveContainer" containerID="bf719317f6721c790e2b52fe684b13c6b4219a93bfee64a5e4e3c2815ee2168c"
Sep 10 17:57:07.378469 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.378450    2549 scope.go:117] "RemoveContainer" containerID="bf719317f6721c790e2b52fe684b13c6b4219a93bfee64a5e4e3c2815ee2168c"
Sep 10 17:57:07.378796 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:57:07.378770    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf719317f6721c790e2b52fe684b13c6b4219a93bfee64a5e4e3c2815ee2168c\": container with ID starting with bf719317f6721c790e2b52fe684b13c6b4219a93bfee64a5e4e3c2815ee2168c not found: ID does not exist" containerID="bf719317f6721c790e2b52fe684b13c6b4219a93bfee64a5e4e3c2815ee2168c"
Sep 10 17:57:07.378883 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.378808    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf719317f6721c790e2b52fe684b13c6b4219a93bfee64a5e4e3c2815ee2168c"} err="failed to get container status \"bf719317f6721c790e2b52fe684b13c6b4219a93bfee64a5e4e3c2815ee2168c\": rpc error: code = NotFound desc = could not find container \"bf719317f6721c790e2b52fe684b13c6b4219a93bfee64a5e4e3c2815ee2168c\": container with ID starting with bf719317f6721c790e2b52fe684b13c6b4219a93bfee64a5e4e3c2815ee2168c not found: ID does not exist"
Sep 10 17:57:07.390329 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.390307    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"]
Sep 10 17:57:07.394356 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.394334    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d0df615c-55c4d687d6-nvcmv"]
Sep 10 17:57:07.493769 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.493725    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"]
Sep 10 17:57:07.494244 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.494131    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="878b5ac3-6c7a-410c-9588-71e33ced3bd8" containerName="maas-api"
Sep 10 17:57:07.494244 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.494152    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="878b5ac3-6c7a-410c-9588-71e33ced3bd8" containerName="maas-api"
Sep 10 17:57:07.494244 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.494166    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="527b2b9f-fd70-4145-8b08-82fbaa63e6c2" containerName="main"
Sep 10 17:57:07.494244 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.494174    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="527b2b9f-fd70-4145-8b08-82fbaa63e6c2" containerName="main"
Sep 10 17:57:07.494244 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.494197    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62539a9b-5ede-4aab-91f8-4dd7242598ac" containerName="maas-api"
Sep 10 17:57:07.494244 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.494205    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="62539a9b-5ede-4aab-91f8-4dd7242598ac" containerName="maas-api"
Sep 10 17:57:07.494572 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.494298    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="878b5ac3-6c7a-410c-9588-71e33ced3bd8" containerName="maas-api"
Sep 10 17:57:07.494572 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.494309    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="62539a9b-5ede-4aab-91f8-4dd7242598ac" containerName="maas-api"
Sep 10 17:57:07.494572 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.494318    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="527b2b9f-fd70-4145-8b08-82fbaa63e6c2" containerName="main"
Sep 10 17:57:07.498846 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.498823    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:07.501858 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.501840    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-c4540c-serving-cert\""
Sep 10 17:57:07.516229 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.516149    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"]
Sep 10 17:57:07.614485 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.614451    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00169d66-6383-4bbe-9b0f-479a3a525795-openshift-service-ca\") pod \"maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:07.614647 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.614500    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00169d66-6383-4bbe-9b0f-479a3a525795-metrics-certs\") pod \"maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:07.614647 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.614603    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00169d66-6383-4bbe-9b0f-479a3a525795-maas-api-tls\") pod \"maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:07.614647 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.614635    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98t95\" (UniqueName: \"kubernetes.io/projected/00169d66-6383-4bbe-9b0f-479a3a525795-kube-api-access-98t95\") pod \"maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:07.715558 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.715511    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00169d66-6383-4bbe-9b0f-479a3a525795-metrics-certs\") pod \"maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:07.715799 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.715606    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00169d66-6383-4bbe-9b0f-479a3a525795-maas-api-tls\") pod \"maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:07.715799 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.715640    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98t95\" (UniqueName: \"kubernetes.io/projected/00169d66-6383-4bbe-9b0f-479a3a525795-kube-api-access-98t95\") pod \"maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:07.715799 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.715723    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00169d66-6383-4bbe-9b0f-479a3a525795-openshift-service-ca\") pod \"maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:07.716370 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.716345    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00169d66-6383-4bbe-9b0f-479a3a525795-openshift-service-ca\") pod \"maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:07.718077 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.718056    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00169d66-6383-4bbe-9b0f-479a3a525795-maas-api-tls\") pod \"maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:07.718175 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.718084    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00169d66-6383-4bbe-9b0f-479a3a525795-metrics-certs\") pod \"maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:07.723961 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.723936    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98t95\" (UniqueName: \"kubernetes.io/projected/00169d66-6383-4bbe-9b0f-479a3a525795-kube-api-access-98t95\") pod \"maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:07.809232 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:07.809192    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:08.035676 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:08.035596    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="878b5ac3-6c7a-410c-9588-71e33ced3bd8" path="/var/lib/kubelet/pods/878b5ac3-6c7a-410c-9588-71e33ced3bd8/volumes"
Sep 10 17:57:08.086615 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:08.086276    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"]
Sep 10 17:57:08.088170 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:57:08.088134    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00169d66_6383_4bbe_9b0f_479a3a525795.slice/crio-55cafb1fd408c45bad9920b9a675e12087b2cc38822a52bca5a94214d63ab9ba WatchSource:0}: Error finding container 55cafb1fd408c45bad9920b9a675e12087b2cc38822a52bca5a94214d63ab9ba: Status 404 returned error can't find the container with id 55cafb1fd408c45bad9920b9a675e12087b2cc38822a52bca5a94214d63ab9ba
Sep 10 17:57:08.376603 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:08.376521    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd" event={"ID":"00169d66-6383-4bbe-9b0f-479a3a525795","Type":"ContainerStarted","Data":"55cafb1fd408c45bad9920b9a675e12087b2cc38822a52bca5a94214d63ab9ba"}
Sep 10 17:57:09.383071 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:09.383036    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd" event={"ID":"00169d66-6383-4bbe-9b0f-479a3a525795","Type":"ContainerStarted","Data":"f29a9b14653839a8c7c7cc9cab4d6f5fedd9fdfa0195af406097c5d943157b4d"}
Sep 10 17:57:09.383466 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:09.383177    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:09.402359 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:09.402303    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd" podStartSLOduration=2.053817668 podStartE2EDuration="2.402283495s" totalImagesPullingTime="348.465827ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:57:09.400069022 +0000 UTC m=+953.891120831" watchObservedRunningTime="2026-09-10 17:57:09.402283495 +0000 UTC m=+953.893335304"
Sep 10 17:57:15.393061 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:15.393032    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:18.502258 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:18.502221    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:57:35.001849 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:35.001411    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:57:41.297611 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.297532    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"]
Sep 10 17:57:41.302551 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.302531    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.306899 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.306870    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-c4540c\"/\"kube-root-ca.crt\""
Sep 10 17:57:41.307083 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.306921    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-c4540c\"/\"default-dockercfg-zdpmv\""
Sep 10 17:57:41.307083 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.306940    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-c4540c\"/\"explicit-model-c4540c-kserve-self-signed-certs\""
Sep 10 17:57:41.307188 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.307069    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-c4540c\"/\"openshift-service-ca.crt\""
Sep 10 17:57:41.312027 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.312004    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"]
Sep 10 17:57:41.423310 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.423268    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvrxv\" (UniqueName: \"kubernetes.io/projected/a78f743b-79e5-4218-b1c6-c51804458ad9-kube-api-access-bvrxv\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.423486 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.423325    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-dshm\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.423486 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.423400    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-model-cache\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.423486 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.423443    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a78f743b-79e5-4218-b1c6-c51804458ad9-tls-certs\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.423486 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.423482    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-tmp-dir\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.423626 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.423515    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-home\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.524327 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.524286    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-model-cache\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.524327 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.524341    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a78f743b-79e5-4218-b1c6-c51804458ad9-tls-certs\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.524587 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.524380    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-tmp-dir\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.524587 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.524404    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-home\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.524587 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.524464    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvrxv\" (UniqueName: \"kubernetes.io/projected/a78f743b-79e5-4218-b1c6-c51804458ad9-kube-api-access-bvrxv\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.524587 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.524508    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-dshm\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.524948 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.524915    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-model-cache\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.525066 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.524971    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-tmp-dir\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.525066 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.524969    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-home\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.526664 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.526643    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-dshm\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.526968 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.526949    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a78f743b-79e5-4218-b1c6-c51804458ad9-tls-certs\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.534449 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.534426    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvrxv\" (UniqueName: \"kubernetes.io/projected/a78f743b-79e5-4218-b1c6-c51804458ad9-kube-api-access-bvrxv\") pod \"explicit-model-c4540c-kserve-68fd64f895-w9n8h\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") " pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.614019 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.613930    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:41.884509 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:41.884482    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"]
Sep 10 17:57:41.886186 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:57:41.886138    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda78f743b_79e5_4218_b1c6_c51804458ad9.slice/crio-bb2ac1dda83ed7ba9afdd4ded2eb6b50fd673a85d99e704d66a8afe35710f8b5 WatchSource:0}: Error finding container bb2ac1dda83ed7ba9afdd4ded2eb6b50fd673a85d99e704d66a8afe35710f8b5: Status 404 returned error can't find the container with id bb2ac1dda83ed7ba9afdd4ded2eb6b50fd673a85d99e704d66a8afe35710f8b5
Sep 10 17:57:42.545365 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:42.545322    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h" event={"ID":"a78f743b-79e5-4218-b1c6-c51804458ad9","Type":"ContainerStarted","Data":"8d29c5148647950ae4e2fe1e85221d8c47606bea957c2d2671e4dd9c4fc4bd23"}
Sep 10 17:57:42.545777 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:42.545377    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h" event={"ID":"a78f743b-79e5-4218-b1c6-c51804458ad9","Type":"ContainerStarted","Data":"bb2ac1dda83ed7ba9afdd4ded2eb6b50fd673a85d99e704d66a8afe35710f8b5"}
Sep 10 17:57:42.564413 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:42.564365    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h" podStartSLOduration=1.564348817 podStartE2EDuration="1.564348817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:57:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:57:42.563811023 +0000 UTC m=+987.054862832" watchObservedRunningTime="2026-09-10 17:57:42.564348817 +0000 UTC m=+987.055400624"
Sep 10 17:57:47.745196 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:47.745155    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"]
Sep 10 17:57:51.614708 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:51.614669    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:57:53.195841 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:53.195799    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:57:58.292805 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.292758    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"]
Sep 10 17:57:58.293292 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.293018    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd" podUID="00169d66-6383-4bbe-9b0f-479a3a525795" containerName="maas-api" containerID="cri-o://f29a9b14653839a8c7c7cc9cab4d6f5fedd9fdfa0195af406097c5d943157b4d" gracePeriod=30
Sep 10 17:57:58.540166 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.540139    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:58.622896 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.622805    2549 generic.go:356] "Generic (PLEG): container finished" podID="00169d66-6383-4bbe-9b0f-479a3a525795" containerID="f29a9b14653839a8c7c7cc9cab4d6f5fedd9fdfa0195af406097c5d943157b4d" exitCode=0
Sep 10 17:57:58.622896 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.622854    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"
Sep 10 17:57:58.623079 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.622885    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd" event={"ID":"00169d66-6383-4bbe-9b0f-479a3a525795","Type":"ContainerDied","Data":"f29a9b14653839a8c7c7cc9cab4d6f5fedd9fdfa0195af406097c5d943157b4d"}
Sep 10 17:57:58.623079 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.622942    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd" event={"ID":"00169d66-6383-4bbe-9b0f-479a3a525795","Type":"ContainerDied","Data":"55cafb1fd408c45bad9920b9a675e12087b2cc38822a52bca5a94214d63ab9ba"}
Sep 10 17:57:58.623079 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.622968    2549 scope.go:117] "RemoveContainer" containerID="f29a9b14653839a8c7c7cc9cab4d6f5fedd9fdfa0195af406097c5d943157b4d"
Sep 10 17:57:58.632027 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.632006    2549 scope.go:117] "RemoveContainer" containerID="f29a9b14653839a8c7c7cc9cab4d6f5fedd9fdfa0195af406097c5d943157b4d"
Sep 10 17:57:58.632350 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:57:58.632330    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f29a9b14653839a8c7c7cc9cab4d6f5fedd9fdfa0195af406097c5d943157b4d\": container with ID starting with f29a9b14653839a8c7c7cc9cab4d6f5fedd9fdfa0195af406097c5d943157b4d not found: ID does not exist" containerID="f29a9b14653839a8c7c7cc9cab4d6f5fedd9fdfa0195af406097c5d943157b4d"
Sep 10 17:57:58.632396 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.632367    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f29a9b14653839a8c7c7cc9cab4d6f5fedd9fdfa0195af406097c5d943157b4d"} err="failed to get container status \"f29a9b14653839a8c7c7cc9cab4d6f5fedd9fdfa0195af406097c5d943157b4d\": rpc error: code = NotFound desc = could not find container \"f29a9b14653839a8c7c7cc9cab4d6f5fedd9fdfa0195af406097c5d943157b4d\": container with ID starting with f29a9b14653839a8c7c7cc9cab4d6f5fedd9fdfa0195af406097c5d943157b4d not found: ID does not exist"
Sep 10 17:57:58.681983 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.681953    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00169d66-6383-4bbe-9b0f-479a3a525795-metrics-certs\") pod \"00169d66-6383-4bbe-9b0f-479a3a525795\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") "
Sep 10 17:57:58.682132 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.681998    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00169d66-6383-4bbe-9b0f-479a3a525795-maas-api-tls\") pod \"00169d66-6383-4bbe-9b0f-479a3a525795\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") "
Sep 10 17:57:58.682132 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.682040    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98t95\" (UniqueName: \"kubernetes.io/projected/00169d66-6383-4bbe-9b0f-479a3a525795-kube-api-access-98t95\") pod \"00169d66-6383-4bbe-9b0f-479a3a525795\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") "
Sep 10 17:57:58.682132 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.682106    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00169d66-6383-4bbe-9b0f-479a3a525795-openshift-service-ca\") pod \"00169d66-6383-4bbe-9b0f-479a3a525795\" (UID: \"00169d66-6383-4bbe-9b0f-479a3a525795\") "
Sep 10 17:57:58.682515 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.682487    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00169d66-6383-4bbe-9b0f-479a3a525795-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "00169d66-6383-4bbe-9b0f-479a3a525795" (UID: "00169d66-6383-4bbe-9b0f-479a3a525795"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:57:58.684282 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.684260    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00169d66-6383-4bbe-9b0f-479a3a525795-kube-api-access-98t95" (OuterVolumeSpecName: "kube-api-access-98t95") pod "00169d66-6383-4bbe-9b0f-479a3a525795" (UID: "00169d66-6383-4bbe-9b0f-479a3a525795"). InnerVolumeSpecName "kube-api-access-98t95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:57:58.684606 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.684584    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00169d66-6383-4bbe-9b0f-479a3a525795-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "00169d66-6383-4bbe-9b0f-479a3a525795" (UID: "00169d66-6383-4bbe-9b0f-479a3a525795"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:57:58.684741 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.684682    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00169d66-6383-4bbe-9b0f-479a3a525795-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "00169d66-6383-4bbe-9b0f-479a3a525795" (UID: "00169d66-6383-4bbe-9b0f-479a3a525795"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:57:58.783289 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.783234    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00169d66-6383-4bbe-9b0f-479a3a525795-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:57:58.783289 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.783285    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00169d66-6383-4bbe-9b0f-479a3a525795-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:57:58.783289 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.783294    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00169d66-6383-4bbe-9b0f-479a3a525795-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:57:58.783289 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.783303    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98t95\" (UniqueName: \"kubernetes.io/projected/00169d66-6383-4bbe-9b0f-479a3a525795-kube-api-access-98t95\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:57:58.946380 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.946350    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"]
Sep 10 17:57:58.949794 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:57:58.949766    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c4540c-6674bd5765-dm8dd"]
Sep 10 17:58:00.029171 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:00.029136    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00169d66-6383-4bbe-9b0f-479a3a525795" path="/var/lib/kubelet/pods/00169d66-6383-4bbe-9b0f-479a3a525795/volumes"
Sep 10 17:58:02.768004 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:02.767940    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h" podUID="a78f743b-79e5-4218-b1c6-c51804458ad9" containerName="main" containerID="cri-o://8d29c5148647950ae4e2fe1e85221d8c47606bea957c2d2671e4dd9c4fc4bd23" gracePeriod=45
Sep 10 17:58:03.020199 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.020139    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:58:03.121641 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.121598    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a78f743b-79e5-4218-b1c6-c51804458ad9-tls-certs\") pod \"a78f743b-79e5-4218-b1c6-c51804458ad9\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") "
Sep 10 17:58:03.121829 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.121655    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-model-cache\") pod \"a78f743b-79e5-4218-b1c6-c51804458ad9\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") "
Sep 10 17:58:03.121829 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.121679    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvrxv\" (UniqueName: \"kubernetes.io/projected/a78f743b-79e5-4218-b1c6-c51804458ad9-kube-api-access-bvrxv\") pod \"a78f743b-79e5-4218-b1c6-c51804458ad9\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") "
Sep 10 17:58:03.121829 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.121729    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-home\") pod \"a78f743b-79e5-4218-b1c6-c51804458ad9\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") "
Sep 10 17:58:03.121829 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.121818    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-tmp-dir\") pod \"a78f743b-79e5-4218-b1c6-c51804458ad9\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") "
Sep 10 17:58:03.122022 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.121858    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-dshm\") pod \"a78f743b-79e5-4218-b1c6-c51804458ad9\" (UID: \"a78f743b-79e5-4218-b1c6-c51804458ad9\") "
Sep 10 17:58:03.122022 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.121911    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-model-cache" (OuterVolumeSpecName: "model-cache") pod "a78f743b-79e5-4218-b1c6-c51804458ad9" (UID: "a78f743b-79e5-4218-b1c6-c51804458ad9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:03.122158 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.122075    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a78f743b-79e5-4218-b1c6-c51804458ad9" (UID: "a78f743b-79e5-4218-b1c6-c51804458ad9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:03.122158 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.122100    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-home" (OuterVolumeSpecName: "home") pod "a78f743b-79e5-4218-b1c6-c51804458ad9" (UID: "a78f743b-79e5-4218-b1c6-c51804458ad9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:03.122270 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.122209    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-tmp-dir\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:03.122270 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.122224    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-model-cache\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:03.122270 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.122236    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-home\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:03.123989 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.123954    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-dshm" (OuterVolumeSpecName: "dshm") pod "a78f743b-79e5-4218-b1c6-c51804458ad9" (UID: "a78f743b-79e5-4218-b1c6-c51804458ad9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:03.124091 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.124036    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a78f743b-79e5-4218-b1c6-c51804458ad9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a78f743b-79e5-4218-b1c6-c51804458ad9" (UID: "a78f743b-79e5-4218-b1c6-c51804458ad9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:58:03.124091 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.124041    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a78f743b-79e5-4218-b1c6-c51804458ad9-kube-api-access-bvrxv" (OuterVolumeSpecName: "kube-api-access-bvrxv") pod "a78f743b-79e5-4218-b1c6-c51804458ad9" (UID: "a78f743b-79e5-4218-b1c6-c51804458ad9"). InnerVolumeSpecName "kube-api-access-bvrxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:58:03.223299 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.223261    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a78f743b-79e5-4218-b1c6-c51804458ad9-dshm\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:03.223299 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.223290    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a78f743b-79e5-4218-b1c6-c51804458ad9-tls-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:03.223299 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.223303    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvrxv\" (UniqueName: \"kubernetes.io/projected/a78f743b-79e5-4218-b1c6-c51804458ad9-kube-api-access-bvrxv\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:03.648531 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.648492    2549 generic.go:356] "Generic (PLEG): container finished" podID="a78f743b-79e5-4218-b1c6-c51804458ad9" containerID="8d29c5148647950ae4e2fe1e85221d8c47606bea957c2d2671e4dd9c4fc4bd23" exitCode=0
Sep 10 17:58:03.648733 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.648541    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"
Sep 10 17:58:03.648733 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.648581    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h" event={"ID":"a78f743b-79e5-4218-b1c6-c51804458ad9","Type":"ContainerDied","Data":"8d29c5148647950ae4e2fe1e85221d8c47606bea957c2d2671e4dd9c4fc4bd23"}
Sep 10 17:58:03.648733 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.648626    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h" event={"ID":"a78f743b-79e5-4218-b1c6-c51804458ad9","Type":"ContainerDied","Data":"bb2ac1dda83ed7ba9afdd4ded2eb6b50fd673a85d99e704d66a8afe35710f8b5"}
Sep 10 17:58:03.648733 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.648643    2549 scope.go:117] "RemoveContainer" containerID="8d29c5148647950ae4e2fe1e85221d8c47606bea957c2d2671e4dd9c4fc4bd23"
Sep 10 17:58:03.657254 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.657235    2549 scope.go:117] "RemoveContainer" containerID="8d29c5148647950ae4e2fe1e85221d8c47606bea957c2d2671e4dd9c4fc4bd23"
Sep 10 17:58:03.657532 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:58:03.657512    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d29c5148647950ae4e2fe1e85221d8c47606bea957c2d2671e4dd9c4fc4bd23\": container with ID starting with 8d29c5148647950ae4e2fe1e85221d8c47606bea957c2d2671e4dd9c4fc4bd23 not found: ID does not exist" containerID="8d29c5148647950ae4e2fe1e85221d8c47606bea957c2d2671e4dd9c4fc4bd23"
Sep 10 17:58:03.657581 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.657544    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d29c5148647950ae4e2fe1e85221d8c47606bea957c2d2671e4dd9c4fc4bd23"} err="failed to get container status \"8d29c5148647950ae4e2fe1e85221d8c47606bea957c2d2671e4dd9c4fc4bd23\": rpc error: code = NotFound desc = could not find container \"8d29c5148647950ae4e2fe1e85221d8c47606bea957c2d2671e4dd9c4fc4bd23\": container with ID starting with 8d29c5148647950ae4e2fe1e85221d8c47606bea957c2d2671e4dd9c4fc4bd23 not found: ID does not exist"
Sep 10 17:58:03.670185 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.670159    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"]
Sep 10 17:58:03.674062 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:03.674038    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-c4540c/explicit-model-c4540c-kserve-68fd64f895-w9n8h"]
Sep 10 17:58:04.029572 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:04.029537    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a78f743b-79e5-4218-b1c6-c51804458ad9" path="/var/lib/kubelet/pods/a78f743b-79e5-4218-b1c6-c51804458ad9/volumes"
Sep 10 17:58:05.400943 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:05.400899    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:58:32.898282 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:32.898243    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:58:39.540983 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:39.540941    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"]
Sep 10 17:58:46.797626 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:46.797588    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:58:48.201357 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.201321    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"]
Sep 10 17:58:48.201727 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.201711    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00169d66-6383-4bbe-9b0f-479a3a525795" containerName="maas-api"
Sep 10 17:58:48.201727 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.201726    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="00169d66-6383-4bbe-9b0f-479a3a525795" containerName="maas-api"
Sep 10 17:58:48.201817 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.201748    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a78f743b-79e5-4218-b1c6-c51804458ad9" containerName="main"
Sep 10 17:58:48.201817 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.201754    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="a78f743b-79e5-4218-b1c6-c51804458ad9" containerName="main"
Sep 10 17:58:48.201880 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.201826    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="00169d66-6383-4bbe-9b0f-479a3a525795" containerName="maas-api"
Sep 10 17:58:48.201880 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.201835    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="a78f743b-79e5-4218-b1c6-c51804458ad9" containerName="main"
Sep 10 17:58:48.204911 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.204894    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:48.207890 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.207864    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1fb38e82-serving-cert\""
Sep 10 17:58:48.213517 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.213448    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"]
Sep 10 17:58:48.341495 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.341450    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2nk8\" (UniqueName: \"kubernetes.io/projected/48054fa7-a302-4e56-ae6f-9617f9d29c90-kube-api-access-j2nk8\") pod \"maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:48.341733 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.341535    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48054fa7-a302-4e56-ae6f-9617f9d29c90-openshift-service-ca\") pod \"maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:48.341733 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.341598    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48054fa7-a302-4e56-ae6f-9617f9d29c90-maas-api-tls\") pod \"maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:48.341733 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.341632    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48054fa7-a302-4e56-ae6f-9617f9d29c90-metrics-certs\") pod \"maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:48.442074 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.442035    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2nk8\" (UniqueName: \"kubernetes.io/projected/48054fa7-a302-4e56-ae6f-9617f9d29c90-kube-api-access-j2nk8\") pod \"maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:48.442270 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.442100    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48054fa7-a302-4e56-ae6f-9617f9d29c90-openshift-service-ca\") pod \"maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:48.442270 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.442175    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48054fa7-a302-4e56-ae6f-9617f9d29c90-maas-api-tls\") pod \"maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:48.442270 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.442211    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48054fa7-a302-4e56-ae6f-9617f9d29c90-metrics-certs\") pod \"maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:48.442464 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:58:48.442321    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-serving-cert: secret "maas-api-e2e-mt-1fb38e82-serving-cert" not found
Sep 10 17:58:48.442464 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:58:48.442384    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48054fa7-a302-4e56-ae6f-9617f9d29c90-maas-api-tls podName:48054fa7-a302-4e56-ae6f-9617f9d29c90 nodeName:}" failed. No retries permitted until 2026-09-10 17:58:48.942367381 +0000 UTC m=+1053.433419167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/48054fa7-a302-4e56-ae6f-9617f9d29c90-maas-api-tls") pod "maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92" (UID: "48054fa7-a302-4e56-ae6f-9617f9d29c90") : secret "maas-api-e2e-mt-1fb38e82-serving-cert" not found
Sep 10 17:58:48.442882 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.442859    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48054fa7-a302-4e56-ae6f-9617f9d29c90-openshift-service-ca\") pod \"maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:48.444536 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.444510    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48054fa7-a302-4e56-ae6f-9617f9d29c90-metrics-certs\") pod \"maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:48.450142 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.450119    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2nk8\" (UniqueName: \"kubernetes.io/projected/48054fa7-a302-4e56-ae6f-9617f9d29c90-kube-api-access-j2nk8\") pod \"maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:48.947136 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.947097    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48054fa7-a302-4e56-ae6f-9617f9d29c90-maas-api-tls\") pod \"maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:48.949516 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:48.949490    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48054fa7-a302-4e56-ae6f-9617f9d29c90-maas-api-tls\") pod \"maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:49.117574 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:49.117539    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:49.396066 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:49.396032    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"]
Sep 10 17:58:49.397158 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:58:49.397129    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48054fa7_a302_4e56_ae6f_9617f9d29c90.slice/crio-81496e28aed18208d4ee5939c024fc50932406b1e7223041abf5f3d5f7e174a1 WatchSource:0}: Error finding container 81496e28aed18208d4ee5939c024fc50932406b1e7223041abf5f3d5f7e174a1: Status 404 returned error can't find the container with id 81496e28aed18208d4ee5939c024fc50932406b1e7223041abf5f3d5f7e174a1
Sep 10 17:58:49.398479 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:49.398459    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:58:49.867680 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:49.867640    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92" event={"ID":"48054fa7-a302-4e56-ae6f-9617f9d29c90","Type":"ContainerStarted","Data":"e38b6da935132733cd0d50d20e1ed314a39ef56f110424fb9a8fd2ec999eb3bd"}
Sep 10 17:58:49.867876 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:49.867710    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92" event={"ID":"48054fa7-a302-4e56-ae6f-9617f9d29c90","Type":"ContainerStarted","Data":"81496e28aed18208d4ee5939c024fc50932406b1e7223041abf5f3d5f7e174a1"}
Sep 10 17:58:49.867876 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:49.867737    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:49.885952 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:49.885897    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92" podStartSLOduration=1.5513771570000001 podStartE2EDuration="1.885880467s" totalImagesPullingTime="334.50331ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:58:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:58:49.883768223 +0000 UTC m=+1054.374820030" watchObservedRunningTime="2026-09-10 17:58:49.885880467 +0000 UTC m=+1054.376932275"
Sep 10 17:58:51.440385 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.440355    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"]
Sep 10 17:58:51.440927 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.440584    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z" podUID="dbae3a46-3569-48c1-aa82-c628fd07651f" containerName="maas-api" containerID="cri-o://2f46e3c1019304aa65e5c222c9392ff70824e89a38764824d141236ab744cb3f" gracePeriod=30
Sep 10 17:58:51.700448 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.700422    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:58:51.774319 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.774283    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbae3a46-3569-48c1-aa82-c628fd07651f-metrics-certs\") pod \"dbae3a46-3569-48c1-aa82-c628fd07651f\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") "
Sep 10 17:58:51.774502 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.774333    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbae3a46-3569-48c1-aa82-c628fd07651f-maas-api-tls\") pod \"dbae3a46-3569-48c1-aa82-c628fd07651f\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") "
Sep 10 17:58:51.774502 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.774367    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlxxv\" (UniqueName: \"kubernetes.io/projected/dbae3a46-3569-48c1-aa82-c628fd07651f-kube-api-access-zlxxv\") pod \"dbae3a46-3569-48c1-aa82-c628fd07651f\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") "
Sep 10 17:58:51.774502 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.774471    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbae3a46-3569-48c1-aa82-c628fd07651f-openshift-service-ca\") pod \"dbae3a46-3569-48c1-aa82-c628fd07651f\" (UID: \"dbae3a46-3569-48c1-aa82-c628fd07651f\") "
Sep 10 17:58:51.774919 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.774885    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbae3a46-3569-48c1-aa82-c628fd07651f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dbae3a46-3569-48c1-aa82-c628fd07651f" (UID: "dbae3a46-3569-48c1-aa82-c628fd07651f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:58:51.776450 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.776419    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbae3a46-3569-48c1-aa82-c628fd07651f-kube-api-access-zlxxv" (OuterVolumeSpecName: "kube-api-access-zlxxv") pod "dbae3a46-3569-48c1-aa82-c628fd07651f" (UID: "dbae3a46-3569-48c1-aa82-c628fd07651f"). InnerVolumeSpecName "kube-api-access-zlxxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:58:51.776561 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.776481    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbae3a46-3569-48c1-aa82-c628fd07651f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "dbae3a46-3569-48c1-aa82-c628fd07651f" (UID: "dbae3a46-3569-48c1-aa82-c628fd07651f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:58:51.776561 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.776498    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbae3a46-3569-48c1-aa82-c628fd07651f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dbae3a46-3569-48c1-aa82-c628fd07651f" (UID: "dbae3a46-3569-48c1-aa82-c628fd07651f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:58:51.876087 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.876053    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbae3a46-3569-48c1-aa82-c628fd07651f-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:51.876087 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.876082    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlxxv\" (UniqueName: \"kubernetes.io/projected/dbae3a46-3569-48c1-aa82-c628fd07651f-kube-api-access-zlxxv\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:51.876087 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.876095    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbae3a46-3569-48c1-aa82-c628fd07651f-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:51.876331 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.876108    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbae3a46-3569-48c1-aa82-c628fd07651f-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:51.878130 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.878101    2549 generic.go:356] "Generic (PLEG): container finished" podID="dbae3a46-3569-48c1-aa82-c628fd07651f" containerID="2f46e3c1019304aa65e5c222c9392ff70824e89a38764824d141236ab744cb3f" exitCode=0
Sep 10 17:58:51.878257 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.878155    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"
Sep 10 17:58:51.878257 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.878187    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z" event={"ID":"dbae3a46-3569-48c1-aa82-c628fd07651f","Type":"ContainerDied","Data":"2f46e3c1019304aa65e5c222c9392ff70824e89a38764824d141236ab744cb3f"}
Sep 10 17:58:51.878257 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.878242    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z" event={"ID":"dbae3a46-3569-48c1-aa82-c628fd07651f","Type":"ContainerDied","Data":"e62406cf7f35e222d03f1689ddd682b66ca33a21abc415f9707fc04d9bfe0805"}
Sep 10 17:58:51.878382 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.878264    2549 scope.go:117] "RemoveContainer" containerID="2f46e3c1019304aa65e5c222c9392ff70824e89a38764824d141236ab744cb3f"
Sep 10 17:58:51.887864 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.887844    2549 scope.go:117] "RemoveContainer" containerID="2f46e3c1019304aa65e5c222c9392ff70824e89a38764824d141236ab744cb3f"
Sep 10 17:58:51.888169 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:58:51.888145    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f46e3c1019304aa65e5c222c9392ff70824e89a38764824d141236ab744cb3f\": container with ID starting with 2f46e3c1019304aa65e5c222c9392ff70824e89a38764824d141236ab744cb3f not found: ID does not exist" containerID="2f46e3c1019304aa65e5c222c9392ff70824e89a38764824d141236ab744cb3f"
Sep 10 17:58:51.888236 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.888182    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f46e3c1019304aa65e5c222c9392ff70824e89a38764824d141236ab744cb3f"} err="failed to get container status \"2f46e3c1019304aa65e5c222c9392ff70824e89a38764824d141236ab744cb3f\": rpc error: code = NotFound desc = could not find container \"2f46e3c1019304aa65e5c222c9392ff70824e89a38764824d141236ab744cb3f\": container with ID starting with 2f46e3c1019304aa65e5c222c9392ff70824e89a38764824d141236ab744cb3f not found: ID does not exist"
Sep 10 17:58:51.900231 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.900203    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"]
Sep 10 17:58:51.904134 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:51.904109    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c34c13-7bfb45884b-46v8z"]
Sep 10 17:58:52.029639 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:52.029543    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbae3a46-3569-48c1-aa82-c628fd07651f" path="/var/lib/kubelet/pods/dbae3a46-3569-48c1-aa82-c628fd07651f/volumes"
Sep 10 17:58:54.562507 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.562460    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz" podUID="46bc48fe-eb5f-44cd-8aca-5a130273f61f" containerName="main" containerID="cri-o://b3263e1997eac7c1fc5fc3433d937bab39a211cb6cdff3602cc818b79071f179" gracePeriod=45
Sep 10 17:58:54.814184 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.814156    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:58:54.901258 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.901220    2549 generic.go:356] "Generic (PLEG): container finished" podID="46bc48fe-eb5f-44cd-8aca-5a130273f61f" containerID="b3263e1997eac7c1fc5fc3433d937bab39a211cb6cdff3602cc818b79071f179" exitCode=0
Sep 10 17:58:54.901401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.901267    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"
Sep 10 17:58:54.901401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.901298    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz" event={"ID":"46bc48fe-eb5f-44cd-8aca-5a130273f61f","Type":"ContainerDied","Data":"b3263e1997eac7c1fc5fc3433d937bab39a211cb6cdff3602cc818b79071f179"}
Sep 10 17:58:54.901401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.901332    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz" event={"ID":"46bc48fe-eb5f-44cd-8aca-5a130273f61f","Type":"ContainerDied","Data":"a6ae94c0a1f6ddd319805a79127670e9294f48b45f648f89f9ff8f416e6130c1"}
Sep 10 17:58:54.901401 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.901348    2549 scope.go:117] "RemoveContainer" containerID="b3263e1997eac7c1fc5fc3433d937bab39a211cb6cdff3602cc818b79071f179"
Sep 10 17:58:54.902092 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.902076    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kncgm\" (UniqueName: \"kubernetes.io/projected/46bc48fe-eb5f-44cd-8aca-5a130273f61f-kube-api-access-kncgm\") pod \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") "
Sep 10 17:58:54.902165 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.902143    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46bc48fe-eb5f-44cd-8aca-5a130273f61f-tls-certs\") pod \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") "
Sep 10 17:58:54.902219 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.902181    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-model-cache\") pod \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") "
Sep 10 17:58:54.902334 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.902290    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-home\") pod \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") "
Sep 10 17:58:54.902393 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.902347    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-dshm\") pod \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") "
Sep 10 17:58:54.902393 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.902376    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-tmp-dir\") pod \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\" (UID: \"46bc48fe-eb5f-44cd-8aca-5a130273f61f\") "
Sep 10 17:58:54.902529 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.902503    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-model-cache" (OuterVolumeSpecName: "model-cache") pod "46bc48fe-eb5f-44cd-8aca-5a130273f61f" (UID: "46bc48fe-eb5f-44cd-8aca-5a130273f61f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:54.902579 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.902517    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-home" (OuterVolumeSpecName: "home") pod "46bc48fe-eb5f-44cd-8aca-5a130273f61f" (UID: "46bc48fe-eb5f-44cd-8aca-5a130273f61f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:54.902768 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.902741    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "46bc48fe-eb5f-44cd-8aca-5a130273f61f" (UID: "46bc48fe-eb5f-44cd-8aca-5a130273f61f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:54.902768 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.902758    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-home\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:54.902908 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.902779    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-model-cache\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:54.904431 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.904409    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46bc48fe-eb5f-44cd-8aca-5a130273f61f-kube-api-access-kncgm" (OuterVolumeSpecName: "kube-api-access-kncgm") pod "46bc48fe-eb5f-44cd-8aca-5a130273f61f" (UID: "46bc48fe-eb5f-44cd-8aca-5a130273f61f"). InnerVolumeSpecName "kube-api-access-kncgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:58:54.904518 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.904445    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-dshm" (OuterVolumeSpecName: "dshm") pod "46bc48fe-eb5f-44cd-8aca-5a130273f61f" (UID: "46bc48fe-eb5f-44cd-8aca-5a130273f61f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:54.904518 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.904475    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46bc48fe-eb5f-44cd-8aca-5a130273f61f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "46bc48fe-eb5f-44cd-8aca-5a130273f61f" (UID: "46bc48fe-eb5f-44cd-8aca-5a130273f61f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:58:54.921549 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.921526    2549 scope.go:117] "RemoveContainer" containerID="b3263e1997eac7c1fc5fc3433d937bab39a211cb6cdff3602cc818b79071f179"
Sep 10 17:58:54.921922 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:58:54.921891    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3263e1997eac7c1fc5fc3433d937bab39a211cb6cdff3602cc818b79071f179\": container with ID starting with b3263e1997eac7c1fc5fc3433d937bab39a211cb6cdff3602cc818b79071f179 not found: ID does not exist" containerID="b3263e1997eac7c1fc5fc3433d937bab39a211cb6cdff3602cc818b79071f179"
Sep 10 17:58:54.922042 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:54.921929    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3263e1997eac7c1fc5fc3433d937bab39a211cb6cdff3602cc818b79071f179"} err="failed to get container status \"b3263e1997eac7c1fc5fc3433d937bab39a211cb6cdff3602cc818b79071f179\": rpc error: code = NotFound desc = could not find container \"b3263e1997eac7c1fc5fc3433d937bab39a211cb6cdff3602cc818b79071f179\": container with ID starting with b3263e1997eac7c1fc5fc3433d937bab39a211cb6cdff3602cc818b79071f179 not found: ID does not exist"
Sep 10 17:58:55.003275 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:55.003191    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kncgm\" (UniqueName: \"kubernetes.io/projected/46bc48fe-eb5f-44cd-8aca-5a130273f61f-kube-api-access-kncgm\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:55.003275 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:55.003223    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46bc48fe-eb5f-44cd-8aca-5a130273f61f-tls-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:55.003275 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:55.003233    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-dshm\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:55.003275 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:55.003243    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46bc48fe-eb5f-44cd-8aca-5a130273f61f-tmp-dir\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:58:55.222883 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:55.222851    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"]
Sep 10 17:58:55.226705 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:55.226660    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-c34c13/test-model-c34c13-kserve-54cdd49d87-9hdzz"]
Sep 10 17:58:55.877868 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:55.877839    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:58:56.029791 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:58:56.029759    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46bc48fe-eb5f-44cd-8aca-5a130273f61f" path="/var/lib/kubelet/pods/46bc48fe-eb5f-44cd-8aca-5a130273f61f/volumes"
Sep 10 17:59:01.703740 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:01.703703    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:59:10.084867 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.084779    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"]
Sep 10 17:59:10.085285 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.084989    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m" podUID="332038d8-86bf-49e3-b5f2-e79743242879" containerName="maas-api" containerID="cri-o://48bd1b64f3cc79808eb0b512bce601b6832a23087b2cab926954f984bbc0a82e" gracePeriod=30
Sep 10 17:59:10.335544 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.335476    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:59:10.439687 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.439650    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/332038d8-86bf-49e3-b5f2-e79743242879-maas-api-tls\") pod \"332038d8-86bf-49e3-b5f2-e79743242879\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") "
Sep 10 17:59:10.439687 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.439714    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/332038d8-86bf-49e3-b5f2-e79743242879-openshift-service-ca\") pod \"332038d8-86bf-49e3-b5f2-e79743242879\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") "
Sep 10 17:59:10.439899 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.439736    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332038d8-86bf-49e3-b5f2-e79743242879-metrics-certs\") pod \"332038d8-86bf-49e3-b5f2-e79743242879\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") "
Sep 10 17:59:10.439899 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.439797    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58vpj\" (UniqueName: \"kubernetes.io/projected/332038d8-86bf-49e3-b5f2-e79743242879-kube-api-access-58vpj\") pod \"332038d8-86bf-49e3-b5f2-e79743242879\" (UID: \"332038d8-86bf-49e3-b5f2-e79743242879\") "
Sep 10 17:59:10.440176 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.440145    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/332038d8-86bf-49e3-b5f2-e79743242879-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "332038d8-86bf-49e3-b5f2-e79743242879" (UID: "332038d8-86bf-49e3-b5f2-e79743242879"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:59:10.441987 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.441953    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/332038d8-86bf-49e3-b5f2-e79743242879-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "332038d8-86bf-49e3-b5f2-e79743242879" (UID: "332038d8-86bf-49e3-b5f2-e79743242879"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:59:10.442073 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.441957    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/332038d8-86bf-49e3-b5f2-e79743242879-kube-api-access-58vpj" (OuterVolumeSpecName: "kube-api-access-58vpj") pod "332038d8-86bf-49e3-b5f2-e79743242879" (UID: "332038d8-86bf-49e3-b5f2-e79743242879"). InnerVolumeSpecName "kube-api-access-58vpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:59:10.442073 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.442006    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/332038d8-86bf-49e3-b5f2-e79743242879-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "332038d8-86bf-49e3-b5f2-e79743242879" (UID: "332038d8-86bf-49e3-b5f2-e79743242879"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:59:10.541193 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.541154    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/332038d8-86bf-49e3-b5f2-e79743242879-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:59:10.541193 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.541186    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/332038d8-86bf-49e3-b5f2-e79743242879-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:59:10.541193 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.541195    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/332038d8-86bf-49e3-b5f2-e79743242879-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:59:10.541193 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.541204    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58vpj\" (UniqueName: \"kubernetes.io/projected/332038d8-86bf-49e3-b5f2-e79743242879-kube-api-access-58vpj\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:59:10.976986 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.976949    2549 generic.go:356] "Generic (PLEG): container finished" podID="332038d8-86bf-49e3-b5f2-e79743242879" containerID="48bd1b64f3cc79808eb0b512bce601b6832a23087b2cab926954f984bbc0a82e" exitCode=0
Sep 10 17:59:10.977167 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.977002    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"
Sep 10 17:59:10.977167 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.977042    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m" event={"ID":"332038d8-86bf-49e3-b5f2-e79743242879","Type":"ContainerDied","Data":"48bd1b64f3cc79808eb0b512bce601b6832a23087b2cab926954f984bbc0a82e"}
Sep 10 17:59:10.977167 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.977100    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m" event={"ID":"332038d8-86bf-49e3-b5f2-e79743242879","Type":"ContainerDied","Data":"d4e19292395111c866a1dfeab9647ad6b47245395bead1d3f1c3470b8f11343f"}
Sep 10 17:59:10.977167 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.977129    2549 scope.go:117] "RemoveContainer" containerID="48bd1b64f3cc79808eb0b512bce601b6832a23087b2cab926954f984bbc0a82e"
Sep 10 17:59:10.986624 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.986602    2549 scope.go:117] "RemoveContainer" containerID="48bd1b64f3cc79808eb0b512bce601b6832a23087b2cab926954f984bbc0a82e"
Sep 10 17:59:10.986992 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:59:10.986971    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48bd1b64f3cc79808eb0b512bce601b6832a23087b2cab926954f984bbc0a82e\": container with ID starting with 48bd1b64f3cc79808eb0b512bce601b6832a23087b2cab926954f984bbc0a82e not found: ID does not exist" containerID="48bd1b64f3cc79808eb0b512bce601b6832a23087b2cab926954f984bbc0a82e"
Sep 10 17:59:10.987052 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.987001    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48bd1b64f3cc79808eb0b512bce601b6832a23087b2cab926954f984bbc0a82e"} err="failed to get container status \"48bd1b64f3cc79808eb0b512bce601b6832a23087b2cab926954f984bbc0a82e\": rpc error: code = NotFound desc = could not find container \"48bd1b64f3cc79808eb0b512bce601b6832a23087b2cab926954f984bbc0a82e\": container with ID starting with 48bd1b64f3cc79808eb0b512bce601b6832a23087b2cab926954f984bbc0a82e not found: ID does not exist"
Sep 10 17:59:10.999778 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:10.999751    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"]
Sep 10 17:59:11.003616 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:11.003592    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-18490e-789d779469-szw8m"]
Sep 10 17:59:12.029531 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:12.029486    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="332038d8-86bf-49e3-b5f2-e79743242879" path="/var/lib/kubelet/pods/332038d8-86bf-49e3-b5f2-e79743242879/volumes"
Sep 10 17:59:13.360723 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:13.360672    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:59:24.993314 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:24.993273    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"]
Sep 10 17:59:24.993711 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:24.993647    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbae3a46-3569-48c1-aa82-c628fd07651f" containerName="maas-api"
Sep 10 17:59:24.993711 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:24.993663    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbae3a46-3569-48c1-aa82-c628fd07651f" containerName="maas-api"
Sep 10 17:59:24.993711 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:24.993673    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="332038d8-86bf-49e3-b5f2-e79743242879" containerName="maas-api"
Sep 10 17:59:24.993711 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:24.993678    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="332038d8-86bf-49e3-b5f2-e79743242879" containerName="maas-api"
Sep 10 17:59:24.993711 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:24.993706    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46bc48fe-eb5f-44cd-8aca-5a130273f61f" containerName="main"
Sep 10 17:59:24.993711 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:24.993712    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="46bc48fe-eb5f-44cd-8aca-5a130273f61f" containerName="main"
Sep 10 17:59:24.993910 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:24.993771    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="332038d8-86bf-49e3-b5f2-e79743242879" containerName="maas-api"
Sep 10 17:59:24.993910 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:24.993781    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="46bc48fe-eb5f-44cd-8aca-5a130273f61f" containerName="main"
Sep 10 17:59:24.993910 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:24.993789    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbae3a46-3569-48c1-aa82-c628fd07651f" containerName="maas-api"
Sep 10 17:59:24.998335 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:24.998314    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:25.000797 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.000774    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-6c9ffc-serving-cert\""
Sep 10 17:59:25.008505 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.008478    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"]
Sep 10 17:59:25.071653 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.071614    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a36e6fe7-bbf7-447c-b15f-f74d38c96988-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:25.071854 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.071765    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a36e6fe7-bbf7-447c-b15f-f74d38c96988-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:25.071854 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.071799    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a36e6fe7-bbf7-447c-b15f-f74d38c96988-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:25.071854 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.071819    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95ksb\" (UniqueName: \"kubernetes.io/projected/a36e6fe7-bbf7-447c-b15f-f74d38c96988-kube-api-access-95ksb\") pod \"maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:25.172475 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.172432    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a36e6fe7-bbf7-447c-b15f-f74d38c96988-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:25.172475 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.172476    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a36e6fe7-bbf7-447c-b15f-f74d38c96988-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:25.172767 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.172497    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95ksb\" (UniqueName: \"kubernetes.io/projected/a36e6fe7-bbf7-447c-b15f-f74d38c96988-kube-api-access-95ksb\") pod \"maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:25.172767 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.172531    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a36e6fe7-bbf7-447c-b15f-f74d38c96988-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:25.173309 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.173285    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a36e6fe7-bbf7-447c-b15f-f74d38c96988-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:25.174952 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.174931    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a36e6fe7-bbf7-447c-b15f-f74d38c96988-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:25.175029 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.174983    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a36e6fe7-bbf7-447c-b15f-f74d38c96988-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:25.180453 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.180422    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95ksb\" (UniqueName: \"kubernetes.io/projected/a36e6fe7-bbf7-447c-b15f-f74d38c96988-kube-api-access-95ksb\") pod \"maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:25.310991 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.310890    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:25.582776 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:25.582746    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"]
Sep 10 17:59:25.585401 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:59:25.585374    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda36e6fe7_bbf7_447c_b15f_f74d38c96988.slice/crio-ab375bd454cfe365496dd48ad057a172645145920b09cf1db6d2ff8943071710 WatchSource:0}: Error finding container ab375bd454cfe365496dd48ad057a172645145920b09cf1db6d2ff8943071710: Status 404 returned error can't find the container with id ab375bd454cfe365496dd48ad057a172645145920b09cf1db6d2ff8943071710
Sep 10 17:59:26.049959 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:26.049916    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm" event={"ID":"a36e6fe7-bbf7-447c-b15f-f74d38c96988","Type":"ContainerStarted","Data":"bb7180a7102f3eb3a0b4490a627178f3f561a096ab0b1db5e6a9307ff904856d"}
Sep 10 17:59:26.050416 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:26.050047    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:26.050416 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:26.050094    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm" event={"ID":"a36e6fe7-bbf7-447c-b15f-f74d38c96988","Type":"ContainerStarted","Data":"ab375bd454cfe365496dd48ad057a172645145920b09cf1db6d2ff8943071710"}
Sep 10 17:59:26.075347 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:26.075291    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm" podStartSLOduration=1.6769600059999998 podStartE2EDuration="2.075271675s" totalImagesPullingTime="398.311669ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:59:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:59:26.07488074 +0000 UTC m=+1090.565932549" watchObservedRunningTime="2026-09-10 17:59:26.075271675 +0000 UTC m=+1090.566323486"
Sep 10 17:59:29.010012 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:29.009969    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:59:29.985442 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:29.985392    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"]
Sep 10 17:59:29.986156 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:29.985757    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92" podUID="48054fa7-a302-4e56-ae6f-9617f9d29c90" containerName="maas-api" containerID="cri-o://e38b6da935132733cd0d50d20e1ed314a39ef56f110424fb9a8fd2ec999eb3bd" gracePeriod=30
Sep 10 17:59:30.244125 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:30.244062    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:59:30.419741 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:30.419674    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2nk8\" (UniqueName: \"kubernetes.io/projected/48054fa7-a302-4e56-ae6f-9617f9d29c90-kube-api-access-j2nk8\") pod \"48054fa7-a302-4e56-ae6f-9617f9d29c90\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") "
Sep 10 17:59:30.419741 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:30.419735    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48054fa7-a302-4e56-ae6f-9617f9d29c90-openshift-service-ca\") pod \"48054fa7-a302-4e56-ae6f-9617f9d29c90\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") "
Sep 10 17:59:30.419951 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:30.419813    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48054fa7-a302-4e56-ae6f-9617f9d29c90-maas-api-tls\") pod \"48054fa7-a302-4e56-ae6f-9617f9d29c90\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") "
Sep 10 17:59:30.419951 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:30.419835    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48054fa7-a302-4e56-ae6f-9617f9d29c90-metrics-certs\") pod \"48054fa7-a302-4e56-ae6f-9617f9d29c90\" (UID: \"48054fa7-a302-4e56-ae6f-9617f9d29c90\") "
Sep 10 17:59:30.420125 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:30.420092    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48054fa7-a302-4e56-ae6f-9617f9d29c90-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "48054fa7-a302-4e56-ae6f-9617f9d29c90" (UID: "48054fa7-a302-4e56-ae6f-9617f9d29c90"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:59:30.421937 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:30.421909    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48054fa7-a302-4e56-ae6f-9617f9d29c90-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "48054fa7-a302-4e56-ae6f-9617f9d29c90" (UID: "48054fa7-a302-4e56-ae6f-9617f9d29c90"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:59:30.422050 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:30.421930    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48054fa7-a302-4e56-ae6f-9617f9d29c90-kube-api-access-j2nk8" (OuterVolumeSpecName: "kube-api-access-j2nk8") pod "48054fa7-a302-4e56-ae6f-9617f9d29c90" (UID: "48054fa7-a302-4e56-ae6f-9617f9d29c90"). InnerVolumeSpecName "kube-api-access-j2nk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:59:30.422050 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:30.421930    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48054fa7-a302-4e56-ae6f-9617f9d29c90-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "48054fa7-a302-4e56-ae6f-9617f9d29c90" (UID: "48054fa7-a302-4e56-ae6f-9617f9d29c90"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:59:30.520589 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:30.520490    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48054fa7-a302-4e56-ae6f-9617f9d29c90-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:59:30.520589 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:30.520527    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48054fa7-a302-4e56-ae6f-9617f9d29c90-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:59:30.520589 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:30.520536    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2nk8\" (UniqueName: \"kubernetes.io/projected/48054fa7-a302-4e56-ae6f-9617f9d29c90-kube-api-access-j2nk8\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:59:30.520589 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:30.520547    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48054fa7-a302-4e56-ae6f-9617f9d29c90-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 17:59:31.074834 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:31.074795    2549 generic.go:356] "Generic (PLEG): container finished" podID="48054fa7-a302-4e56-ae6f-9617f9d29c90" containerID="e38b6da935132733cd0d50d20e1ed314a39ef56f110424fb9a8fd2ec999eb3bd" exitCode=0
Sep 10 17:59:31.075014 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:31.074844    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92" event={"ID":"48054fa7-a302-4e56-ae6f-9617f9d29c90","Type":"ContainerDied","Data":"e38b6da935132733cd0d50d20e1ed314a39ef56f110424fb9a8fd2ec999eb3bd"}
Sep 10 17:59:31.075014 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:31.074851    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"
Sep 10 17:59:31.075014 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:31.074885    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92" event={"ID":"48054fa7-a302-4e56-ae6f-9617f9d29c90","Type":"ContainerDied","Data":"81496e28aed18208d4ee5939c024fc50932406b1e7223041abf5f3d5f7e174a1"}
Sep 10 17:59:31.075014 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:31.074908    2549 scope.go:117] "RemoveContainer" containerID="e38b6da935132733cd0d50d20e1ed314a39ef56f110424fb9a8fd2ec999eb3bd"
Sep 10 17:59:31.084492 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:31.084474    2549 scope.go:117] "RemoveContainer" containerID="e38b6da935132733cd0d50d20e1ed314a39ef56f110424fb9a8fd2ec999eb3bd"
Sep 10 17:59:31.084796 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:59:31.084776    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e38b6da935132733cd0d50d20e1ed314a39ef56f110424fb9a8fd2ec999eb3bd\": container with ID starting with e38b6da935132733cd0d50d20e1ed314a39ef56f110424fb9a8fd2ec999eb3bd not found: ID does not exist" containerID="e38b6da935132733cd0d50d20e1ed314a39ef56f110424fb9a8fd2ec999eb3bd"
Sep 10 17:59:31.084853 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:31.084806    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e38b6da935132733cd0d50d20e1ed314a39ef56f110424fb9a8fd2ec999eb3bd"} err="failed to get container status \"e38b6da935132733cd0d50d20e1ed314a39ef56f110424fb9a8fd2ec999eb3bd\": rpc error: code = NotFound desc = could not find container \"e38b6da935132733cd0d50d20e1ed314a39ef56f110424fb9a8fd2ec999eb3bd\": container with ID starting with e38b6da935132733cd0d50d20e1ed314a39ef56f110424fb9a8fd2ec999eb3bd not found: ID does not exist"
Sep 10 17:59:31.097065 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:31.097034    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"]
Sep 10 17:59:31.099594 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:31.099571    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1fb38e82-5d87c68b5d-whb92"]
Sep 10 17:59:32.029570 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:32.029532    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48054fa7-a302-4e56-ae6f-9617f9d29c90" path="/var/lib/kubelet/pods/48054fa7-a302-4e56-ae6f-9617f9d29c90/volumes"
Sep 10 17:59:32.061143 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:32.061115    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 17:59:39.414260 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:39.414225    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 17:59:53.507542 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.507505    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"]
Sep 10 17:59:53.508085 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.508044    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48054fa7-a302-4e56-ae6f-9617f9d29c90" containerName="maas-api"
Sep 10 17:59:53.508085 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.508059    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="48054fa7-a302-4e56-ae6f-9617f9d29c90" containerName="maas-api"
Sep 10 17:59:53.508182 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.508170    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="48054fa7-a302-4e56-ae6f-9617f9d29c90" containerName="maas-api"
Sep 10 17:59:53.511474 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.511455    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:53.514232 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.514208    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-fc23cf-serving-cert\""
Sep 10 17:59:53.520027 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.520002    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"]
Sep 10 17:59:53.614793 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.614750    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4652b83c-f542-4587-ad05-6dbe47fc4fb2-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:53.614988 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.614803    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2w2s\" (UniqueName: \"kubernetes.io/projected/4652b83c-f542-4587-ad05-6dbe47fc4fb2-kube-api-access-x2w2s\") pod \"maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:53.614988 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.614864    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4652b83c-f542-4587-ad05-6dbe47fc4fb2-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:53.614988 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.614926    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4652b83c-f542-4587-ad05-6dbe47fc4fb2-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:53.715626 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.715586    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4652b83c-f542-4587-ad05-6dbe47fc4fb2-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:53.715626 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.715625    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2w2s\" (UniqueName: \"kubernetes.io/projected/4652b83c-f542-4587-ad05-6dbe47fc4fb2-kube-api-access-x2w2s\") pod \"maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:53.715879 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.715720    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4652b83c-f542-4587-ad05-6dbe47fc4fb2-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:53.715879 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.715783    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4652b83c-f542-4587-ad05-6dbe47fc4fb2-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:53.715960 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:59:53.715893    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-serving-cert: secret "maas-api-e2e-shared-b-w3-fc23cf-serving-cert" not found
Sep 10 17:59:53.715998 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:59:53.715971    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4652b83c-f542-4587-ad05-6dbe47fc4fb2-maas-api-tls podName:4652b83c-f542-4587-ad05-6dbe47fc4fb2 nodeName:}" failed. No retries permitted until 2026-09-10 17:59:54.215948057 +0000 UTC m=+1118.706999858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4652b83c-f542-4587-ad05-6dbe47fc4fb2-maas-api-tls") pod "maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96" (UID: "4652b83c-f542-4587-ad05-6dbe47fc4fb2") : secret "maas-api-e2e-shared-b-w3-fc23cf-serving-cert" not found
Sep 10 17:59:53.716440 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.716413    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4652b83c-f542-4587-ad05-6dbe47fc4fb2-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:53.718138 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.718116    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4652b83c-f542-4587-ad05-6dbe47fc4fb2-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:53.725962 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:53.725937    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2w2s\" (UniqueName: \"kubernetes.io/projected/4652b83c-f542-4587-ad05-6dbe47fc4fb2-kube-api-access-x2w2s\") pod \"maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:54.219918 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:54.219881    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4652b83c-f542-4587-ad05-6dbe47fc4fb2-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:54.222411 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:54.222380    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4652b83c-f542-4587-ad05-6dbe47fc4fb2-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:54.423189 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:54.423147    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:54.694224 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:54.694195    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"]
Sep 10 17:59:54.696529 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:59:54.696503    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4652b83c_f542_4587_ad05_6dbe47fc4fb2.slice/crio-4712962f729a4b61775406389d2872dcd883c526c20cb22e7279f9f56bd5a82a WatchSource:0}: Error finding container 4712962f729a4b61775406389d2872dcd883c526c20cb22e7279f9f56bd5a82a: Status 404 returned error can't find the container with id 4712962f729a4b61775406389d2872dcd883c526c20cb22e7279f9f56bd5a82a
Sep 10 17:59:55.193323 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.193287    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96" event={"ID":"4652b83c-f542-4587-ad05-6dbe47fc4fb2","Type":"ContainerStarted","Data":"d18e3be67ecd18d5f0014e8e6ce95d3d83a7d299a4811fce33347d51b2693078"}
Sep 10 17:59:55.193500 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.193341    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96" event={"ID":"4652b83c-f542-4587-ad05-6dbe47fc4fb2","Type":"ContainerStarted","Data":"4712962f729a4b61775406389d2872dcd883c526c20cb22e7279f9f56bd5a82a"}
Sep 10 17:59:55.193500 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.193389    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 17:59:55.211031 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.210981    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96" podStartSLOduration=1.873850531 podStartE2EDuration="2.210963978s" totalImagesPullingTime="337.113447ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:59:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:59:55.21007724 +0000 UTC m=+1119.701129061" watchObservedRunningTime="2026-09-10 17:59:55.210963978 +0000 UTC m=+1119.702015786"
Sep 10 17:59:55.428397 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.428315    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"]
Sep 10 17:59:55.431705 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.431672    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:55.434386 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.434363    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-971a94-serving-cert\""
Sep 10 17:59:55.446791 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.446756    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"]
Sep 10 17:59:55.532342 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.532307    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5679e79-95ee-47ef-8254-2a03514661dd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:55.532342 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.532344    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5679e79-95ee-47ef-8254-2a03514661dd-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:55.532607 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.532369    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c6kx\" (UniqueName: \"kubernetes.io/projected/a5679e79-95ee-47ef-8254-2a03514661dd-kube-api-access-8c6kx\") pod \"maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:55.532607 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.532457    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5679e79-95ee-47ef-8254-2a03514661dd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:55.633927 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.633880    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5679e79-95ee-47ef-8254-2a03514661dd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:55.633927 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.633928    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5679e79-95ee-47ef-8254-2a03514661dd-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:55.634169 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.633951    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c6kx\" (UniqueName: \"kubernetes.io/projected/a5679e79-95ee-47ef-8254-2a03514661dd-kube-api-access-8c6kx\") pod \"maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:55.634169 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.633981    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5679e79-95ee-47ef-8254-2a03514661dd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:55.634169 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:59:55.634088    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-serving-cert: secret "maas-api-e2e-shared-a-w2-971a94-serving-cert" not found
Sep 10 17:59:55.634169 ip-10-0-139-245 kubenswrapper[2549]: E0910 17:59:55.634157    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5679e79-95ee-47ef-8254-2a03514661dd-maas-api-tls podName:a5679e79-95ee-47ef-8254-2a03514661dd nodeName:}" failed. No retries permitted until 2026-09-10 17:59:56.134130103 +0000 UTC m=+1120.625181890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a5679e79-95ee-47ef-8254-2a03514661dd-maas-api-tls") pod "maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997" (UID: "a5679e79-95ee-47ef-8254-2a03514661dd") : secret "maas-api-e2e-shared-a-w2-971a94-serving-cert" not found
Sep 10 17:59:55.634645 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.634622    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5679e79-95ee-47ef-8254-2a03514661dd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:55.636286 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.636264    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5679e79-95ee-47ef-8254-2a03514661dd-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:55.643120 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:55.643098    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c6kx\" (UniqueName: \"kubernetes.io/projected/a5679e79-95ee-47ef-8254-2a03514661dd-kube-api-access-8c6kx\") pod \"maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:56.145019 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:56.144958    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5679e79-95ee-47ef-8254-2a03514661dd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:56.147375 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:56.147353    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5679e79-95ee-47ef-8254-2a03514661dd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:56.341787 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:56.341739    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:56.616017 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:56.615807    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"]
Sep 10 17:59:56.618228 ip-10-0-139-245 kubenswrapper[2549]: W0910 17:59:56.618200    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5679e79_95ee_47ef_8254_2a03514661dd.slice/crio-19338ab841fbad2d9a6105db8d1fbf726f78921a63ab1d1c3104f1def3277e36 WatchSource:0}: Error finding container 19338ab841fbad2d9a6105db8d1fbf726f78921a63ab1d1c3104f1def3277e36: Status 404 returned error can't find the container with id 19338ab841fbad2d9a6105db8d1fbf726f78921a63ab1d1c3104f1def3277e36
Sep 10 17:59:57.203807 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:57.203767    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997" event={"ID":"a5679e79-95ee-47ef-8254-2a03514661dd","Type":"ContainerStarted","Data":"9c452bc0a0cad20957ebbe8e6e244d21fcd754ffcc36d954f724ed23fb356a04"}
Sep 10 17:59:57.204195 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:57.203824    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997" event={"ID":"a5679e79-95ee-47ef-8254-2a03514661dd","Type":"ContainerStarted","Data":"19338ab841fbad2d9a6105db8d1fbf726f78921a63ab1d1c3104f1def3277e36"}
Sep 10 17:59:57.204195 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:57.203851    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 17:59:57.222106 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:57.222003    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997" podStartSLOduration=1.883753995 podStartE2EDuration="2.221987247s" totalImagesPullingTime="338.233252ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:59:57.220668421 +0000 UTC m=+1121.711720230" watchObservedRunningTime="2026-09-10 17:59:57.221987247 +0000 UTC m=+1121.713039054"
Sep 10 17:59:58.903523 ip-10-0-139-245 kubenswrapper[2549]: I0910 17:59:58.903478    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:00:01.203754 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:01.203687    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 18:00:03.213899 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:03.213867    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 18:00:23.848232 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:23.848196    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"]
Sep 10 18:00:23.853265 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:23.853244    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:23.856014 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:23.855989    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-09fbb5-serving-cert\""
Sep 10 18:00:23.867471 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:23.867444    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"]
Sep 10 18:00:23.913814 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:23.913769    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:23.913814 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:23.913824    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:23.914028 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:23.913900    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwcml\" (UniqueName: \"kubernetes.io/projected/ad87b84f-a336-4503-84d4-fd1761b65c4b-kube-api-access-qwcml\") pod \"maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:23.914028 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:23.913943    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad87b84f-a336-4503-84d4-fd1761b65c4b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:24.015156 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:24.015097    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:24.015315 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:24.015194    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:24.015315 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:24.015255    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwcml\" (UniqueName: \"kubernetes.io/projected/ad87b84f-a336-4503-84d4-fd1761b65c4b-kube-api-access-qwcml\") pod \"maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:24.015315 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:24.015297    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad87b84f-a336-4503-84d4-fd1761b65c4b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:24.015454 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:00:24.015333    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-serving-cert: secret "maas-api-e2e-shared-b-w2-09fbb5-serving-cert" not found
Sep 10 18:00:24.015454 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:00:24.015404    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-maas-api-tls podName:ad87b84f-a336-4503-84d4-fd1761b65c4b nodeName:}" failed. No retries permitted until 2026-09-10 18:00:24.515386236 +0000 UTC m=+1149.006438021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-maas-api-tls") pod "maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz" (UID: "ad87b84f-a336-4503-84d4-fd1761b65c4b") : secret "maas-api-e2e-shared-b-w2-09fbb5-serving-cert" not found
Sep 10 18:00:24.016007 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:24.015986    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad87b84f-a336-4503-84d4-fd1761b65c4b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:24.017686 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:24.017664    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:24.038138 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:24.038104    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwcml\" (UniqueName: \"kubernetes.io/projected/ad87b84f-a336-4503-84d4-fd1761b65c4b-kube-api-access-qwcml\") pod \"maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:24.518622 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:24.518582    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:24.519043 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:00:24.518757    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-serving-cert: secret "maas-api-e2e-shared-b-w2-09fbb5-serving-cert" not found
Sep 10 18:00:24.519043 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:00:24.518821    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-maas-api-tls podName:ad87b84f-a336-4503-84d4-fd1761b65c4b nodeName:}" failed. No retries permitted until 2026-09-10 18:00:25.518806739 +0000 UTC m=+1150.009858525 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-maas-api-tls") pod "maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz" (UID: "ad87b84f-a336-4503-84d4-fd1761b65c4b") : secret "maas-api-e2e-shared-b-w2-09fbb5-serving-cert" not found
Sep 10 18:00:25.528725 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:25.528674    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:25.531292 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:25.531269    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:25.663355 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:25.663318    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:25.935890 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:25.935862    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"]
Sep 10 18:00:25.938446 ip-10-0-139-245 kubenswrapper[2549]: W0910 18:00:25.938417    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad87b84f_a336_4503_84d4_fd1761b65c4b.slice/crio-fa0a0c2f56160f8329cc072ae0da87c9b584ae9d86b6ec1c2d308f91adccc903 WatchSource:0}: Error finding container fa0a0c2f56160f8329cc072ae0da87c9b584ae9d86b6ec1c2d308f91adccc903: Status 404 returned error can't find the container with id fa0a0c2f56160f8329cc072ae0da87c9b584ae9d86b6ec1c2d308f91adccc903
Sep 10 18:00:26.340707 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:26.340666    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz" event={"ID":"ad87b84f-a336-4503-84d4-fd1761b65c4b","Type":"ContainerStarted","Data":"fa0a0c2f56160f8329cc072ae0da87c9b584ae9d86b6ec1c2d308f91adccc903"}
Sep 10 18:00:27.346685 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:27.346646    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz" event={"ID":"ad87b84f-a336-4503-84d4-fd1761b65c4b","Type":"ContainerStarted","Data":"c617b88b3def5c8fd601711710da4e4daf8ac5633579b4a9f398b669a737ce9b"}
Sep 10 18:00:27.347133 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:27.346768    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:27.367520 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:27.367471    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz" podStartSLOduration=3.961510156 podStartE2EDuration="4.367454893s" totalImagesPullingTime="405.944737ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:00:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:00:27.365883644 +0000 UTC m=+1151.856935449" watchObservedRunningTime="2026-09-10 18:00:27.367454893 +0000 UTC m=+1151.858506723"
Sep 10 18:00:30.402213 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:30.402171    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:00:33.357153 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.357118    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:00:33.407209 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.407174    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"]
Sep 10 18:00:33.411843 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.411819    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.414608 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.414583    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-6c9ffc\"/\"openshift-service-ca.crt\""
Sep 10 18:00:33.414727 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.414616    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-6c9ffc\"/\"default-dockercfg-5b6b4\""
Sep 10 18:00:33.414794 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.414769    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-6c9ffc\"/\"kube-root-ca.crt\""
Sep 10 18:00:33.414899 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.414881    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-6c9ffc\"/\"auth-test-model-6c9ffc-kserve-self-signed-certs\""
Sep 10 18:00:33.420530 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.420504    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"]
Sep 10 18:00:33.501743 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.501686    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-model-cache\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.501902 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.501762    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcwbt\" (UniqueName: \"kubernetes.io/projected/b5ed7f66-ac33-4b7c-8e75-876ae7806331-kube-api-access-gcwbt\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.501902 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.501792    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5ed7f66-ac33-4b7c-8e75-876ae7806331-tls-certs\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.501902 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.501841    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-home\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.501902 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.501879    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-dshm\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.501902 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.501901    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-tmp-dir\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.603034 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.602998    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-model-cache\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.603147 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.603041    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcwbt\" (UniqueName: \"kubernetes.io/projected/b5ed7f66-ac33-4b7c-8e75-876ae7806331-kube-api-access-gcwbt\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.603147 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.603070    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5ed7f66-ac33-4b7c-8e75-876ae7806331-tls-certs\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.603147 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.603116    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-home\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.603147 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.603136    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-dshm\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.603301 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.603159    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-tmp-dir\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.603511 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.603490    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-model-cache\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.603561 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.603515    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-home\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.603597 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.603558    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-tmp-dir\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.605386 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.605358    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-dshm\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.605683 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.605663    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5ed7f66-ac33-4b7c-8e75-876ae7806331-tls-certs\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.612856 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.612795    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcwbt\" (UniqueName: \"kubernetes.io/projected/b5ed7f66-ac33-4b7c-8e75-876ae7806331-kube-api-access-gcwbt\") pod \"auth-test-model-6c9ffc-kserve-79b4987779-vkvj6\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.722713 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.722647    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:33.997591 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:33.997561    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"]
Sep 10 18:00:33.999511 ip-10-0-139-245 kubenswrapper[2549]: W0910 18:00:33.999485    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5ed7f66_ac33_4b7c_8e75_876ae7806331.slice/crio-c31c5559afcbbac2922c4b945353416cae92c29d7fc656e85edc8f702a61d72b WatchSource:0}: Error finding container c31c5559afcbbac2922c4b945353416cae92c29d7fc656e85edc8f702a61d72b: Status 404 returned error can't find the container with id c31c5559afcbbac2922c4b945353416cae92c29d7fc656e85edc8f702a61d72b
Sep 10 18:00:34.382584 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:34.382486    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6" event={"ID":"b5ed7f66-ac33-4b7c-8e75-876ae7806331","Type":"ContainerStarted","Data":"b40edd8397542488f6e831fb9600ff11336d8e240da456edc64168c8a823e854"}
Sep 10 18:00:34.382584 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:34.382536    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6" event={"ID":"b5ed7f66-ac33-4b7c-8e75-876ae7806331","Type":"ContainerStarted","Data":"c31c5559afcbbac2922c4b945353416cae92c29d7fc656e85edc8f702a61d72b"}
Sep 10 18:00:34.405654 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:34.405597    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6" podStartSLOduration=1.405579764 podStartE2EDuration="1.405579764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:00:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:00:34.404172545 +0000 UTC m=+1158.895224354" watchObservedRunningTime="2026-09-10 18:00:34.405579764 +0000 UTC m=+1158.896631572"
Sep 10 18:00:43.723097 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:43.723006    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:43.723618 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:43.723167    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:43.735711 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:43.735669    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:44.437096 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:44.437061    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:00:46.002976 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:00:46.002936    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:01:00.847730 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:00.847677    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"]
Sep 10 18:01:00.852281 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:00.852256    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:00.855068 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:00.855034    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-be08bfb3-serving-cert\""
Sep 10 18:01:00.865090 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:00.865055    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"]
Sep 10 18:01:00.961843 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:00.961791    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-openshift-service-ca\") pod \"maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:00.962030 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:00.961854    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jvmd\" (UniqueName: \"kubernetes.io/projected/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-kube-api-access-2jvmd\") pod \"maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:00.962030 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:00.961949    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-maas-api-tls\") pod \"maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:00.962030 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:00.962012    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-metrics-certs\") pod \"maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:01.063262 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:01.063222    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-metrics-certs\") pod \"maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:01.063456 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:01.063287    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-openshift-service-ca\") pod \"maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:01.063456 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:01.063318    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jvmd\" (UniqueName: \"kubernetes.io/projected/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-kube-api-access-2jvmd\") pod \"maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:01.063456 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:01.063352    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-maas-api-tls\") pod \"maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:01.063456 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:01:01.063452    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-serving-cert: secret "maas-api-e2e-mt-be08bfb3-serving-cert" not found
Sep 10 18:01:01.063731 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:01:01.063527    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-maas-api-tls podName:6493d4f2-9f02-4919-99a4-3b1c3f9df0b8 nodeName:}" failed. No retries permitted until 2026-09-10 18:01:01.56350761 +0000 UTC m=+1186.054559410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-maas-api-tls") pod "maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv" (UID: "6493d4f2-9f02-4919-99a4-3b1c3f9df0b8") : secret "maas-api-e2e-mt-be08bfb3-serving-cert" not found
Sep 10 18:01:01.064040 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:01.064017    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-openshift-service-ca\") pod \"maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:01.065856 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:01.065826    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-metrics-certs\") pod \"maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:01.073938 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:01.073910    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jvmd\" (UniqueName: \"kubernetes.io/projected/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-kube-api-access-2jvmd\") pod \"maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:01.568967 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:01.568924    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-maas-api-tls\") pod \"maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:01.571521 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:01.571493    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-maas-api-tls\") pod \"maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:01.763747 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:01.763702    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:02.046609 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:02.045901    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"]
Sep 10 18:01:02.051711 ip-10-0-139-245 kubenswrapper[2549]: W0910 18:01:02.048411    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6493d4f2_9f02_4919_99a4_3b1c3f9df0b8.slice/crio-f031c63af4c10b695b4f8aebe5b322a58029dc7dca51f54111d3260bd2b20621 WatchSource:0}: Error finding container f031c63af4c10b695b4f8aebe5b322a58029dc7dca51f54111d3260bd2b20621: Status 404 returned error can't find the container with id f031c63af4c10b695b4f8aebe5b322a58029dc7dca51f54111d3260bd2b20621
Sep 10 18:01:02.118603 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:02.118564    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:01:02.513720 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:02.513672    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv" event={"ID":"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8","Type":"ContainerStarted","Data":"f031c63af4c10b695b4f8aebe5b322a58029dc7dca51f54111d3260bd2b20621"}
Sep 10 18:01:03.519641 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:03.519597    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv" event={"ID":"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8","Type":"ContainerStarted","Data":"cd86d9ffd69f267bc7b4502cd807058873c823c9e4eb64dd7e8c310c5169a9c4"}
Sep 10 18:01:03.520066 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:03.519722    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:03.546247 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:03.546198    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv" podStartSLOduration=3.132274591 podStartE2EDuration="3.546182183s" totalImagesPullingTime="413.907592ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:01:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:01:03.543854084 +0000 UTC m=+1188.034905892" watchObservedRunningTime="2026-09-10 18:01:03.546182183 +0000 UTC m=+1188.037234024"
Sep 10 18:01:09.529736 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:09.529683    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:01:19.711832 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:19.711786    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:01:31.216043 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:31.216000    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:01:47.517578 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:01:47.517534    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:02:06.910440 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:06.910396    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:02:19.482277 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.482036    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"]
Sep 10 18:02:19.482978 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.482326    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv" podUID="6493d4f2-9f02-4919-99a4-3b1c3f9df0b8" containerName="maas-api" containerID="cri-o://cd86d9ffd69f267bc7b4502cd807058873c823c9e4eb64dd7e8c310c5169a9c4" gracePeriod=30
Sep 10 18:02:19.525069 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.525024    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv" podUID="6493d4f2-9f02-4919-99a4-3b1c3f9df0b8" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.48:8443/health\": dial tcp 10.134.0.48:8443: connect: connection refused"
Sep 10 18:02:19.741742 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.741655    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:02:19.804395 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.804362    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-metrics-certs\") pod \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") "
Sep 10 18:02:19.804395 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.804399    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-maas-api-tls\") pod \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") "
Sep 10 18:02:19.804649 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.804424    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-openshift-service-ca\") pod \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") "
Sep 10 18:02:19.804649 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.804576    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jvmd\" (UniqueName: \"kubernetes.io/projected/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-kube-api-access-2jvmd\") pod \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\" (UID: \"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8\") "
Sep 10 18:02:19.804878 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.804853    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6493d4f2-9f02-4919-99a4-3b1c3f9df0b8" (UID: "6493d4f2-9f02-4919-99a4-3b1c3f9df0b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:02:19.806655 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.806620    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6493d4f2-9f02-4919-99a4-3b1c3f9df0b8" (UID: "6493d4f2-9f02-4919-99a4-3b1c3f9df0b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:02:19.806794 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.806678    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6493d4f2-9f02-4919-99a4-3b1c3f9df0b8" (UID: "6493d4f2-9f02-4919-99a4-3b1c3f9df0b8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:02:19.806794 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.806745    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-kube-api-access-2jvmd" (OuterVolumeSpecName: "kube-api-access-2jvmd") pod "6493d4f2-9f02-4919-99a4-3b1c3f9df0b8" (UID: "6493d4f2-9f02-4919-99a4-3b1c3f9df0b8"). InnerVolumeSpecName "kube-api-access-2jvmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:02:19.873477 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.873440    2549 generic.go:356] "Generic (PLEG): container finished" podID="6493d4f2-9f02-4919-99a4-3b1c3f9df0b8" containerID="cd86d9ffd69f267bc7b4502cd807058873c823c9e4eb64dd7e8c310c5169a9c4" exitCode=0
Sep 10 18:02:19.873646 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.873485    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"
Sep 10 18:02:19.873646 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.873528    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv" event={"ID":"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8","Type":"ContainerDied","Data":"cd86d9ffd69f267bc7b4502cd807058873c823c9e4eb64dd7e8c310c5169a9c4"}
Sep 10 18:02:19.873646 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.873577    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv" event={"ID":"6493d4f2-9f02-4919-99a4-3b1c3f9df0b8","Type":"ContainerDied","Data":"f031c63af4c10b695b4f8aebe5b322a58029dc7dca51f54111d3260bd2b20621"}
Sep 10 18:02:19.873646 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.873602    2549 scope.go:117] "RemoveContainer" containerID="cd86d9ffd69f267bc7b4502cd807058873c823c9e4eb64dd7e8c310c5169a9c4"
Sep 10 18:02:19.883570 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.883550    2549 scope.go:117] "RemoveContainer" containerID="cd86d9ffd69f267bc7b4502cd807058873c823c9e4eb64dd7e8c310c5169a9c4"
Sep 10 18:02:19.883928 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:02:19.883908    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd86d9ffd69f267bc7b4502cd807058873c823c9e4eb64dd7e8c310c5169a9c4\": container with ID starting with cd86d9ffd69f267bc7b4502cd807058873c823c9e4eb64dd7e8c310c5169a9c4 not found: ID does not exist" containerID="cd86d9ffd69f267bc7b4502cd807058873c823c9e4eb64dd7e8c310c5169a9c4"
Sep 10 18:02:19.883988 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.883937    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd86d9ffd69f267bc7b4502cd807058873c823c9e4eb64dd7e8c310c5169a9c4"} err="failed to get container status \"cd86d9ffd69f267bc7b4502cd807058873c823c9e4eb64dd7e8c310c5169a9c4\": rpc error: code = NotFound desc = could not find container \"cd86d9ffd69f267bc7b4502cd807058873c823c9e4eb64dd7e8c310c5169a9c4\": container with ID starting with cd86d9ffd69f267bc7b4502cd807058873c823c9e4eb64dd7e8c310c5169a9c4 not found: ID does not exist"
Sep 10 18:02:19.900212 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.900174    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"]
Sep 10 18:02:19.903359 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.903330    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be08bfb3-5874b7c7dc-q45bv"]
Sep 10 18:02:19.905853 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.905839    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:02:19.905906 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.905858    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:02:19.905906 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.905867    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:02:19.905906 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:19.905877    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jvmd\" (UniqueName: \"kubernetes.io/projected/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8-kube-api-access-2jvmd\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:02:20.029852 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:20.029768    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6493d4f2-9f02-4919-99a4-3b1c3f9df0b8" path="/var/lib/kubelet/pods/6493d4f2-9f02-4919-99a4-3b1c3f9df0b8/volumes"
Sep 10 18:02:29.711874 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:29.711833    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:02:47.220332 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:02:47.220297    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:03:00.903907 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:00.903868    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:03:18.203447 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:18.203409    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:03:23.380220 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:23.380184    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"]
Sep 10 18:03:27.008039 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.008000    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"]
Sep 10 18:03:27.008576 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.008555    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6493d4f2-9f02-4919-99a4-3b1c3f9df0b8" containerName="maas-api"
Sep 10 18:03:27.008658 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.008579    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="6493d4f2-9f02-4919-99a4-3b1c3f9df0b8" containerName="maas-api"
Sep 10 18:03:27.008735 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.008713    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="6493d4f2-9f02-4919-99a4-3b1c3f9df0b8" containerName="maas-api"
Sep 10 18:03:27.012794 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.012774    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.016368 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.016347    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-6c9ffc\"/\"sub-test-model-6c9ffc-kserve-self-signed-certs\""
Sep 10 18:03:27.024010 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.023987    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"]
Sep 10 18:03:27.144397 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.144355    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-home\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.144579 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.144426    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-tls-certs\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.144579 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.144459    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-tmp-dir\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.144579 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.144487    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn9lp\" (UniqueName: \"kubernetes.io/projected/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-kube-api-access-xn9lp\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.144579 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.144542    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-model-cache\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.144762 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.144586    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-dshm\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.245484 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.245440    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-dshm\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.245669 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.245517    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-home\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.245669 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.245583    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-tls-certs\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.245669 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.245604    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-tmp-dir\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.245669 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.245637    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn9lp\" (UniqueName: \"kubernetes.io/projected/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-kube-api-access-xn9lp\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.245669 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.245659    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-model-cache\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.246066 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.245960    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-home\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.246066 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.246017    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-tmp-dir\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.246066 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.246050    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-model-cache\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.247772 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.247753    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-dshm\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.248295 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.248270    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-tls-certs\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.264348 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.264284    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn9lp\" (UniqueName: \"kubernetes.io/projected/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-kube-api-access-xn9lp\") pod \"sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") " pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.322348 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.322313    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:27.600364 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:27.600334    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"]
Sep 10 18:03:27.602829 ip-10-0-139-245 kubenswrapper[2549]: W0910 18:03:27.602799    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37ad96e1_92b3_4bce_b554_4f1ec6d48f02.slice/crio-be28d1cb1d5bf11c72b49a7d492f593f55b7f101ed10dfa639e8d5f1f8cf0da5 WatchSource:0}: Error finding container be28d1cb1d5bf11c72b49a7d492f593f55b7f101ed10dfa639e8d5f1f8cf0da5: Status 404 returned error can't find the container with id be28d1cb1d5bf11c72b49a7d492f593f55b7f101ed10dfa639e8d5f1f8cf0da5
Sep 10 18:03:28.199529 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:28.199493    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn" event={"ID":"37ad96e1-92b3-4bce-b554-4f1ec6d48f02","Type":"ContainerStarted","Data":"e0ca11e18dde72bfe26eba799ad846bf734d90c48f46b6456587735da4fc8a24"}
Sep 10 18:03:28.199529 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:28.199534    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn" event={"ID":"37ad96e1-92b3-4bce-b554-4f1ec6d48f02","Type":"ContainerStarted","Data":"be28d1cb1d5bf11c72b49a7d492f593f55b7f101ed10dfa639e8d5f1f8cf0da5"}
Sep 10 18:03:28.221292 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:28.221243    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn" podStartSLOduration=2.221227104 podStartE2EDuration="2.221227104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:03:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:03:28.218719757 +0000 UTC m=+1332.709771568" watchObservedRunningTime="2026-09-10 18:03:28.221227104 +0000 UTC m=+1332.712278911"
Sep 10 18:03:37.117187 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:37.117146    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:03:37.323013 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:37.322963    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:37.323225 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:37.323029    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:37.340821 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:37.340786    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:38.263240 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.263157    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:03:38.400520 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.400471    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6" podUID="b5ed7f66-ac33-4b7c-8e75-876ae7806331" containerName="main" containerID="cri-o://b40edd8397542488f6e831fb9600ff11336d8e240da456edc64168c8a823e854" gracePeriod=45
Sep 10 18:03:38.647268 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.647239    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:03:38.655740 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.655715    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5ed7f66-ac33-4b7c-8e75-876ae7806331-tls-certs\") pod \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") "
Sep 10 18:03:38.655828 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.655749    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-dshm\") pod \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") "
Sep 10 18:03:38.655828 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.655792    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-model-cache\") pod \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") "
Sep 10 18:03:38.655924 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.655830    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcwbt\" (UniqueName: \"kubernetes.io/projected/b5ed7f66-ac33-4b7c-8e75-876ae7806331-kube-api-access-gcwbt\") pod \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") "
Sep 10 18:03:38.655924 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.655865    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-tmp-dir\") pod \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") "
Sep 10 18:03:38.655924 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.655898    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-home\") pod \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\" (UID: \"b5ed7f66-ac33-4b7c-8e75-876ae7806331\") "
Sep 10 18:03:38.656163 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.656097    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-model-cache" (OuterVolumeSpecName: "model-cache") pod "b5ed7f66-ac33-4b7c-8e75-876ae7806331" (UID: "b5ed7f66-ac33-4b7c-8e75-876ae7806331"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:03:38.656243 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.656196    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b5ed7f66-ac33-4b7c-8e75-876ae7806331" (UID: "b5ed7f66-ac33-4b7c-8e75-876ae7806331"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:03:38.656299 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.656254    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-home" (OuterVolumeSpecName: "home") pod "b5ed7f66-ac33-4b7c-8e75-876ae7806331" (UID: "b5ed7f66-ac33-4b7c-8e75-876ae7806331"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:03:38.658010 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.657985    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5ed7f66-ac33-4b7c-8e75-876ae7806331-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b5ed7f66-ac33-4b7c-8e75-876ae7806331" (UID: "b5ed7f66-ac33-4b7c-8e75-876ae7806331"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:03:38.658114 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.658057    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5ed7f66-ac33-4b7c-8e75-876ae7806331-kube-api-access-gcwbt" (OuterVolumeSpecName: "kube-api-access-gcwbt") pod "b5ed7f66-ac33-4b7c-8e75-876ae7806331" (UID: "b5ed7f66-ac33-4b7c-8e75-876ae7806331"). InnerVolumeSpecName "kube-api-access-gcwbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:03:38.658114 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.658072    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-dshm" (OuterVolumeSpecName: "dshm") pod "b5ed7f66-ac33-4b7c-8e75-876ae7806331" (UID: "b5ed7f66-ac33-4b7c-8e75-876ae7806331"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:03:38.757349 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.757302    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5ed7f66-ac33-4b7c-8e75-876ae7806331-tls-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:03:38.757349 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.757337    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-dshm\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:03:38.757349 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.757349    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-model-cache\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:03:38.757349 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.757360    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcwbt\" (UniqueName: \"kubernetes.io/projected/b5ed7f66-ac33-4b7c-8e75-876ae7806331-kube-api-access-gcwbt\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:03:38.757641 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.757397    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-tmp-dir\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:03:38.757641 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:38.757410    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5ed7f66-ac33-4b7c-8e75-876ae7806331-home\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:03:39.252194 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:39.252156    2549 generic.go:356] "Generic (PLEG): container finished" podID="b5ed7f66-ac33-4b7c-8e75-876ae7806331" containerID="b40edd8397542488f6e831fb9600ff11336d8e240da456edc64168c8a823e854" exitCode=0
Sep 10 18:03:39.252194 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:39.252199    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"
Sep 10 18:03:39.252461 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:39.252238    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6" event={"ID":"b5ed7f66-ac33-4b7c-8e75-876ae7806331","Type":"ContainerDied","Data":"b40edd8397542488f6e831fb9600ff11336d8e240da456edc64168c8a823e854"}
Sep 10 18:03:39.252461 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:39.252279    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6" event={"ID":"b5ed7f66-ac33-4b7c-8e75-876ae7806331","Type":"ContainerDied","Data":"c31c5559afcbbac2922c4b945353416cae92c29d7fc656e85edc8f702a61d72b"}
Sep 10 18:03:39.252461 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:39.252301    2549 scope.go:117] "RemoveContainer" containerID="b40edd8397542488f6e831fb9600ff11336d8e240da456edc64168c8a823e854"
Sep 10 18:03:39.261326 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:39.261309    2549 scope.go:117] "RemoveContainer" containerID="b40edd8397542488f6e831fb9600ff11336d8e240da456edc64168c8a823e854"
Sep 10 18:03:39.261591 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:03:39.261573    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b40edd8397542488f6e831fb9600ff11336d8e240da456edc64168c8a823e854\": container with ID starting with b40edd8397542488f6e831fb9600ff11336d8e240da456edc64168c8a823e854 not found: ID does not exist" containerID="b40edd8397542488f6e831fb9600ff11336d8e240da456edc64168c8a823e854"
Sep 10 18:03:39.261635 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:39.261600    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b40edd8397542488f6e831fb9600ff11336d8e240da456edc64168c8a823e854"} err="failed to get container status \"b40edd8397542488f6e831fb9600ff11336d8e240da456edc64168c8a823e854\": rpc error: code = NotFound desc = could not find container \"b40edd8397542488f6e831fb9600ff11336d8e240da456edc64168c8a823e854\": container with ID starting with b40edd8397542488f6e831fb9600ff11336d8e240da456edc64168c8a823e854 not found: ID does not exist"
Sep 10 18:03:39.287433 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:39.287400    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"]
Sep 10 18:03:39.296144 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:39.296119    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6c9ffc/auth-test-model-6c9ffc-kserve-79b4987779-vkvj6"]
Sep 10 18:03:40.029280 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:40.029243    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5ed7f66-ac33-4b7c-8e75-876ae7806331" path="/var/lib/kubelet/pods/b5ed7f66-ac33-4b7c-8e75-876ae7806331/volumes"
Sep 10 18:03:53.425903 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:03:53.425865    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:04:11.968088 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:11.968053    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:04:12.227845 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.227769    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"]
Sep 10 18:04:12.228186 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.228173    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5ed7f66-ac33-4b7c-8e75-876ae7806331" containerName="main"
Sep 10 18:04:12.228228 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.228187    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5ed7f66-ac33-4b7c-8e75-876ae7806331" containerName="main"
Sep 10 18:04:12.228278 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.228269    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5ed7f66-ac33-4b7c-8e75-876ae7806331" containerName="main"
Sep 10 18:04:12.236276 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.236248    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:12.239636 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.239609    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-b38800-serving-cert\""
Sep 10 18:04:12.242659 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.242634    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"]
Sep 10 18:04:12.364817 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.364779    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-metrics-certs\") pod \"maas-api-e2e-api-a-b38800-6d7799b785-w469j\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:12.365005 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.364849    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-maas-api-tls\") pod \"maas-api-e2e-api-a-b38800-6d7799b785-w469j\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:12.365005 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.364946    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-openshift-service-ca\") pod \"maas-api-e2e-api-a-b38800-6d7799b785-w469j\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:12.365101 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.365016    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlh42\" (UniqueName: \"kubernetes.io/projected/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-kube-api-access-xlh42\") pod \"maas-api-e2e-api-a-b38800-6d7799b785-w469j\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:12.465538 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.465506    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-metrics-certs\") pod \"maas-api-e2e-api-a-b38800-6d7799b785-w469j\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:12.465752 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.465560    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-maas-api-tls\") pod \"maas-api-e2e-api-a-b38800-6d7799b785-w469j\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:12.465752 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.465590    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-openshift-service-ca\") pod \"maas-api-e2e-api-a-b38800-6d7799b785-w469j\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:12.465752 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.465629    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlh42\" (UniqueName: \"kubernetes.io/projected/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-kube-api-access-xlh42\") pod \"maas-api-e2e-api-a-b38800-6d7799b785-w469j\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:12.466380 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.466352    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-openshift-service-ca\") pod \"maas-api-e2e-api-a-b38800-6d7799b785-w469j\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:12.468090 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.468066    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-metrics-certs\") pod \"maas-api-e2e-api-a-b38800-6d7799b785-w469j\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:12.468184 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.468120    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-maas-api-tls\") pod \"maas-api-e2e-api-a-b38800-6d7799b785-w469j\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:12.474856 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.474836    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlh42\" (UniqueName: \"kubernetes.io/projected/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-kube-api-access-xlh42\") pod \"maas-api-e2e-api-a-b38800-6d7799b785-w469j\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:12.547964 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.547873    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:12.826403 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.826377    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"]
Sep 10 18:04:12.828603 ip-10-0-139-245 kubenswrapper[2549]: W0910 18:04:12.828571    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd782431_d0e6_44d6_9da3_4e00ecedfbbe.slice/crio-00c3ce4a225d52638f37acd144135922c9c1ed4e69a2588b459de87333423fa4 WatchSource:0}: Error finding container 00c3ce4a225d52638f37acd144135922c9c1ed4e69a2588b459de87333423fa4: Status 404 returned error can't find the container with id 00c3ce4a225d52638f37acd144135922c9c1ed4e69a2588b459de87333423fa4
Sep 10 18:04:12.830484 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:12.830464    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 18:04:13.418908 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:13.418870    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j" event={"ID":"bd782431-d0e6-44d6-9da3-4e00ecedfbbe","Type":"ContainerStarted","Data":"3f40aa7425b7e33e835d7bed4d7007bd2b5cba280fd62dbd8fc659de2807f537"}
Sep 10 18:04:13.419348 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:13.418923    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j" event={"ID":"bd782431-d0e6-44d6-9da3-4e00ecedfbbe","Type":"ContainerStarted","Data":"00c3ce4a225d52638f37acd144135922c9c1ed4e69a2588b459de87333423fa4"}
Sep 10 18:04:13.419348 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:13.419010    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:13.443756 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:13.443671    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j" podStartSLOduration=1.005848857 podStartE2EDuration="1.443651605s" totalImagesPullingTime="437.802748ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:04:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:04:13.442798743 +0000 UTC m=+1377.933850553" watchObservedRunningTime="2026-09-10 18:04:13.443651605 +0000 UTC m=+1377.934703412"
Sep 10 18:04:19.430462 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:19.430431    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:04:25.321643 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:25.321608    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7791-e33cf8"]
Sep 10 18:04:25.326353 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:25.326332    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-e33cf8"
Sep 10 18:04:25.330014 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:25.329992    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-r67rg\""
Sep 10 18:04:25.336774 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:25.336749    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7791-e33cf8"]
Sep 10 18:04:25.385408 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:25.385373    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2t7c\" (UniqueName: \"kubernetes.io/projected/6013e8d1-af8c-4a4a-b930-e9a650fba714-kube-api-access-h2t7c\") pod \"test-curl-7791-e33cf8\" (UID: \"6013e8d1-af8c-4a4a-b930-e9a650fba714\") " pod="opendatahub/test-curl-7791-e33cf8"
Sep 10 18:04:25.486750 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:25.486679    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2t7c\" (UniqueName: \"kubernetes.io/projected/6013e8d1-af8c-4a4a-b930-e9a650fba714-kube-api-access-h2t7c\") pod \"test-curl-7791-e33cf8\" (UID: \"6013e8d1-af8c-4a4a-b930-e9a650fba714\") " pod="opendatahub/test-curl-7791-e33cf8"
Sep 10 18:04:25.499245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:25.499205    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2t7c\" (UniqueName: \"kubernetes.io/projected/6013e8d1-af8c-4a4a-b930-e9a650fba714-kube-api-access-h2t7c\") pod \"test-curl-7791-e33cf8\" (UID: \"6013e8d1-af8c-4a4a-b930-e9a650fba714\") " pod="opendatahub/test-curl-7791-e33cf8"
Sep 10 18:04:25.635533 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:25.635441    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-e33cf8"
Sep 10 18:04:25.909274 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:25.909247    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7791-e33cf8"]
Sep 10 18:04:25.911229 ip-10-0-139-245 kubenswrapper[2549]: W0910 18:04:25.911198    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6013e8d1_af8c_4a4a_b930_e9a650fba714.slice/crio-17f5923885cf4f8716114c3b3628216e0b921fae1d681ef38d7b745d2e390786 WatchSource:0}: Error finding container 17f5923885cf4f8716114c3b3628216e0b921fae1d681ef38d7b745d2e390786: Status 404 returned error can't find the container with id 17f5923885cf4f8716114c3b3628216e0b921fae1d681ef38d7b745d2e390786
Sep 10 18:04:26.484944 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:26.484910    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7791-e33cf8" event={"ID":"6013e8d1-af8c-4a4a-b930-e9a650fba714","Type":"ContainerStarted","Data":"17f5923885cf4f8716114c3b3628216e0b921fae1d681ef38d7b745d2e390786"}
Sep 10 18:04:28.433140 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:28.433108    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:04:29.501054 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:29.501013    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7791-e33cf8" event={"ID":"6013e8d1-af8c-4a4a-b930-e9a650fba714","Type":"ContainerStarted","Data":"f767c0c6fcae6d5ed908f9731eaee207d30251f99580d26b0b30d5ce4985e546"}
Sep 10 18:04:29.524174 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:29.524120    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7791-e33cf8" podStartSLOduration=1.96411668 podStartE2EDuration="4.524105808s" totalImagesPullingTime="2.559989128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:04:29.521557827 +0000 UTC m=+1394.012609636" watchObservedRunningTime="2026-09-10 18:04:29.524105808 +0000 UTC m=+1394.015157646"
Sep 10 18:04:30.211429 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:30.211381    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7791-e33cf8"]
Sep 10 18:04:30.216232 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:30.216204    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7791-e33cf8"]
Sep 10 18:04:30.338708 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:30.338601    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7791-1450e1"]
Sep 10 18:04:30.339055 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:30.339035    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6013e8d1-af8c-4a4a-b930-e9a650fba714" containerName="test-curl-7791-e33cf8"
Sep 10 18:04:30.339055 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:30.339052    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="6013e8d1-af8c-4a4a-b930-e9a650fba714" containerName="test-curl-7791-e33cf8"
Sep 10 18:04:30.339233 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:30.339149    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="6013e8d1-af8c-4a4a-b930-e9a650fba714" containerName="test-curl-7791-e33cf8"
Sep 10 18:04:30.342235 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:30.342217    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-1450e1"
Sep 10 18:04:30.369461 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:30.369410    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7791-1450e1"]
Sep 10 18:04:30.432537 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:30.432496    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6gn5\" (UniqueName: \"kubernetes.io/projected/70cb23b9-81ed-4e69-9f66-70fab866de25-kube-api-access-d6gn5\") pod \"test-curl-7791-1450e1\" (UID: \"70cb23b9-81ed-4e69-9f66-70fab866de25\") " pod="opendatahub/test-curl-7791-1450e1"
Sep 10 18:04:30.506932 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:04:30.506898    2549 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7791-e33cf8" container="test-curl-7791-e33cf8"
Sep 10 18:04:30.533611 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:30.533571    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6gn5\" (UniqueName: \"kubernetes.io/projected/70cb23b9-81ed-4e69-9f66-70fab866de25-kube-api-access-d6gn5\") pod \"test-curl-7791-1450e1\" (UID: \"70cb23b9-81ed-4e69-9f66-70fab866de25\") " pod="opendatahub/test-curl-7791-1450e1"
Sep 10 18:04:30.547049 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:30.547017    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6gn5\" (UniqueName: \"kubernetes.io/projected/70cb23b9-81ed-4e69-9f66-70fab866de25-kube-api-access-d6gn5\") pod \"test-curl-7791-1450e1\" (UID: \"70cb23b9-81ed-4e69-9f66-70fab866de25\") " pod="opendatahub/test-curl-7791-1450e1"
Sep 10 18:04:30.651396 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:30.651309    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-1450e1"
Sep 10 18:04:30.926281 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:30.926253    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7791-1450e1"]
Sep 10 18:04:30.928623 ip-10-0-139-245 kubenswrapper[2549]: W0910 18:04:30.928595    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70cb23b9_81ed_4e69_9f66_70fab866de25.slice/crio-c92b06b994f9a913ce2167384240d30c92636e6d9bd56138b1735b30b3562d83 WatchSource:0}: Error finding container c92b06b994f9a913ce2167384240d30c92636e6d9bd56138b1735b30b3562d83: Status 404 returned error can't find the container with id c92b06b994f9a913ce2167384240d30c92636e6d9bd56138b1735b30b3562d83
Sep 10 18:04:31.513192 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:31.513152    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7791-1450e1" event={"ID":"70cb23b9-81ed-4e69-9f66-70fab866de25","Type":"ContainerStarted","Data":"e0ee35c88a9a933f4471c4a2873254a16d5887732d5134144c01f60077060eae"}
Sep 10 18:04:31.513192 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:31.513197    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7791-1450e1" event={"ID":"70cb23b9-81ed-4e69-9f66-70fab866de25","Type":"ContainerStarted","Data":"c92b06b994f9a913ce2167384240d30c92636e6d9bd56138b1735b30b3562d83"}
Sep 10 18:04:31.513735 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:31.513295    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7791-e33cf8" podUID="6013e8d1-af8c-4a4a-b930-e9a650fba714" containerName="test-curl-7791-e33cf8" containerID="cri-o://f767c0c6fcae6d5ed908f9731eaee207d30251f99580d26b0b30d5ce4985e546" gracePeriod=2
Sep 10 18:04:31.543090 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:31.543035    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7791-1450e1" podStartSLOduration=1.543020106 podStartE2EDuration="1.543020106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:04:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:04:31.542032522 +0000 UTC m=+1396.033084341" watchObservedRunningTime="2026-09-10 18:04:31.543020106 +0000 UTC m=+1396.034071914"
Sep 10 18:04:31.545594 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:04:31.545568    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7791-e33cf8\" is forbidden: User \"system:node:ip-10-0-139-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-245.ec2.internal' and this object" podUID="6013e8d1-af8c-4a4a-b930-e9a650fba714" pod="opendatahub/test-curl-7791-e33cf8"
Sep 10 18:04:32.158447 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:32.158408    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7791-1450e1"]
Sep 10 18:04:32.186548 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:32.186508    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7791-1450e1"]
Sep 10 18:04:32.517497 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:04:32.517458    2549 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7791-1450e1" container="test-curl-7791-1450e1"
Sep 10 18:04:33.522840 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:33.522782    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7791-1450e1" podUID="70cb23b9-81ed-4e69-9f66-70fab866de25" containerName="test-curl-7791-1450e1" containerID="cri-o://e0ee35c88a9a933f4471c4a2873254a16d5887732d5134144c01f60077060eae" gracePeriod=2
Sep 10 18:04:33.526522 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:04:33.526494    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7791-1450e1\" is forbidden: User \"system:node:ip-10-0-139-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-245.ec2.internal' and this object" podUID="70cb23b9-81ed-4e69-9f66-70fab866de25" pod="opendatahub/test-curl-7791-1450e1"
Sep 10 18:04:33.683609 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:33.683583    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-e33cf8"
Sep 10 18:04:33.686267 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:04:33.686235    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7791-1450e1\" is forbidden: User \"system:node:ip-10-0-139-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-245.ec2.internal' and this object" podUID="70cb23b9-81ed-4e69-9f66-70fab866de25" pod="opendatahub/test-curl-7791-1450e1"
Sep 10 18:04:33.763506 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:33.763465    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2t7c\" (UniqueName: \"kubernetes.io/projected/6013e8d1-af8c-4a4a-b930-e9a650fba714-kube-api-access-h2t7c\") pod \"6013e8d1-af8c-4a4a-b930-e9a650fba714\" (UID: \"6013e8d1-af8c-4a4a-b930-e9a650fba714\") "
Sep 10 18:04:33.765708 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:33.765665    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6013e8d1-af8c-4a4a-b930-e9a650fba714-kube-api-access-h2t7c" (OuterVolumeSpecName: "kube-api-access-h2t7c") pod "6013e8d1-af8c-4a4a-b930-e9a650fba714" (UID: "6013e8d1-af8c-4a4a-b930-e9a650fba714"). InnerVolumeSpecName "kube-api-access-h2t7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:04:33.865060 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:33.865021    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2t7c\" (UniqueName: \"kubernetes.io/projected/6013e8d1-af8c-4a4a-b930-e9a650fba714-kube-api-access-h2t7c\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:04:34.029945 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:34.029848    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6013e8d1-af8c-4a4a-b930-e9a650fba714" path="/var/lib/kubelet/pods/6013e8d1-af8c-4a4a-b930-e9a650fba714/volumes"
Sep 10 18:04:34.364274 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:34.364189    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"]
Sep 10 18:04:34.528310 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:34.528273    2549 generic.go:356] "Generic (PLEG): container finished" podID="6013e8d1-af8c-4a4a-b930-e9a650fba714" containerID="f767c0c6fcae6d5ed908f9731eaee207d30251f99580d26b0b30d5ce4985e546" exitCode=137
Sep 10 18:04:34.528310 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:34.528312    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-e33cf8"
Sep 10 18:04:34.528817 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:34.528384    2549 scope.go:117] "RemoveContainer" containerID="f767c0c6fcae6d5ed908f9731eaee207d30251f99580d26b0b30d5ce4985e546"
Sep 10 18:04:34.536885 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:34.536863    2549 scope.go:117] "RemoveContainer" containerID="f767c0c6fcae6d5ed908f9731eaee207d30251f99580d26b0b30d5ce4985e546"
Sep 10 18:04:34.537147 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:04:34.537127    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f767c0c6fcae6d5ed908f9731eaee207d30251f99580d26b0b30d5ce4985e546\": container with ID starting with f767c0c6fcae6d5ed908f9731eaee207d30251f99580d26b0b30d5ce4985e546 not found: ID does not exist" containerID="f767c0c6fcae6d5ed908f9731eaee207d30251f99580d26b0b30d5ce4985e546"
Sep 10 18:04:34.537198 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:34.537156    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f767c0c6fcae6d5ed908f9731eaee207d30251f99580d26b0b30d5ce4985e546"} err="failed to get container status \"f767c0c6fcae6d5ed908f9731eaee207d30251f99580d26b0b30d5ce4985e546\": rpc error: code = NotFound desc = could not find container \"f767c0c6fcae6d5ed908f9731eaee207d30251f99580d26b0b30d5ce4985e546\": container with ID starting with f767c0c6fcae6d5ed908f9731eaee207d30251f99580d26b0b30d5ce4985e546 not found: ID does not exist"
Sep 10 18:04:35.698195 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:35.698165    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-1450e1"
Sep 10 18:04:35.783850 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:35.783760    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6gn5\" (UniqueName: \"kubernetes.io/projected/70cb23b9-81ed-4e69-9f66-70fab866de25-kube-api-access-d6gn5\") pod \"70cb23b9-81ed-4e69-9f66-70fab866de25\" (UID: \"70cb23b9-81ed-4e69-9f66-70fab866de25\") "
Sep 10 18:04:35.785875 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:35.785842    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70cb23b9-81ed-4e69-9f66-70fab866de25-kube-api-access-d6gn5" (OuterVolumeSpecName: "kube-api-access-d6gn5") pod "70cb23b9-81ed-4e69-9f66-70fab866de25" (UID: "70cb23b9-81ed-4e69-9f66-70fab866de25"). InnerVolumeSpecName "kube-api-access-d6gn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:04:35.885032 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:35.884996    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6gn5\" (UniqueName: \"kubernetes.io/projected/70cb23b9-81ed-4e69-9f66-70fab866de25-kube-api-access-d6gn5\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:04:36.030022 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:36.029985    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70cb23b9-81ed-4e69-9f66-70fab866de25" path="/var/lib/kubelet/pods/70cb23b9-81ed-4e69-9f66-70fab866de25/volumes"
Sep 10 18:04:36.540612 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:36.540572    2549 generic.go:356] "Generic (PLEG): container finished" podID="70cb23b9-81ed-4e69-9f66-70fab866de25" containerID="e0ee35c88a9a933f4471c4a2873254a16d5887732d5134144c01f60077060eae" exitCode=137
Sep 10 18:04:36.540612 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:36.540606    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7791-1450e1"
Sep 10 18:04:36.540870 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:36.540645    2549 scope.go:117] "RemoveContainer" containerID="e0ee35c88a9a933f4471c4a2873254a16d5887732d5134144c01f60077060eae"
Sep 10 18:04:36.549404 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:36.549382    2549 scope.go:117] "RemoveContainer" containerID="e0ee35c88a9a933f4471c4a2873254a16d5887732d5134144c01f60077060eae"
Sep 10 18:04:36.549726 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:04:36.549687    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0ee35c88a9a933f4471c4a2873254a16d5887732d5134144c01f60077060eae\": container with ID starting with e0ee35c88a9a933f4471c4a2873254a16d5887732d5134144c01f60077060eae not found: ID does not exist" containerID="e0ee35c88a9a933f4471c4a2873254a16d5887732d5134144c01f60077060eae"
Sep 10 18:04:36.549811 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:36.549736    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0ee35c88a9a933f4471c4a2873254a16d5887732d5134144c01f60077060eae"} err="failed to get container status \"e0ee35c88a9a933f4471c4a2873254a16d5887732d5134144c01f60077060eae\": rpc error: code = NotFound desc = could not find container \"e0ee35c88a9a933f4471c4a2873254a16d5887732d5134144c01f60077060eae\": container with ID starting with e0ee35c88a9a933f4471c4a2873254a16d5887732d5134144c01f60077060eae not found: ID does not exist"
Sep 10 18:04:41.405017 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.404981    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"]
Sep 10 18:04:41.405379 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.405361    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70cb23b9-81ed-4e69-9f66-70fab866de25" containerName="test-curl-7791-1450e1"
Sep 10 18:04:41.405379 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.405373    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="70cb23b9-81ed-4e69-9f66-70fab866de25" containerName="test-curl-7791-1450e1"
Sep 10 18:04:41.405450 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.405445    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="70cb23b9-81ed-4e69-9f66-70fab866de25" containerName="test-curl-7791-1450e1"
Sep 10 18:04:41.409875 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.409851    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:41.416497 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.416477    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-d439f7-serving-cert\""
Sep 10 18:04:41.423564 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.423516    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"]
Sep 10 18:04:41.543653 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.543614    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faede3d1-b97e-4904-b14e-edc6d8bd6d92-metrics-certs\") pod \"maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:41.543653 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.543667    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrws2\" (UniqueName: \"kubernetes.io/projected/faede3d1-b97e-4904-b14e-edc6d8bd6d92-kube-api-access-lrws2\") pod \"maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:41.543931 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.543778    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faede3d1-b97e-4904-b14e-edc6d8bd6d92-openshift-service-ca\") pod \"maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:41.543931 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.543870    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faede3d1-b97e-4904-b14e-edc6d8bd6d92-maas-api-tls\") pod \"maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:41.644402 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.644358    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faede3d1-b97e-4904-b14e-edc6d8bd6d92-openshift-service-ca\") pod \"maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:41.644608 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.644418    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faede3d1-b97e-4904-b14e-edc6d8bd6d92-maas-api-tls\") pod \"maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:41.644608 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.644471    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faede3d1-b97e-4904-b14e-edc6d8bd6d92-metrics-certs\") pod \"maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:41.644608 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.644500    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrws2\" (UniqueName: \"kubernetes.io/projected/faede3d1-b97e-4904-b14e-edc6d8bd6d92-kube-api-access-lrws2\") pod \"maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:41.644863 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:04:41.644602    2549 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-serving-cert: secret "maas-api-e2e-api-b-d439f7-serving-cert" not found
Sep 10 18:04:41.644863 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:04:41.644683    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faede3d1-b97e-4904-b14e-edc6d8bd6d92-maas-api-tls podName:faede3d1-b97e-4904-b14e-edc6d8bd6d92 nodeName:}" failed. No retries permitted until 2026-09-10 18:04:42.144659735 +0000 UTC m=+1406.635711525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/faede3d1-b97e-4904-b14e-edc6d8bd6d92-maas-api-tls") pod "maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj" (UID: "faede3d1-b97e-4904-b14e-edc6d8bd6d92") : secret "maas-api-e2e-api-b-d439f7-serving-cert" not found
Sep 10 18:04:41.645068 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.645041    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faede3d1-b97e-4904-b14e-edc6d8bd6d92-openshift-service-ca\") pod \"maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:41.646951 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.646932    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faede3d1-b97e-4904-b14e-edc6d8bd6d92-metrics-certs\") pod \"maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:41.657791 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:41.657739    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrws2\" (UniqueName: \"kubernetes.io/projected/faede3d1-b97e-4904-b14e-edc6d8bd6d92-kube-api-access-lrws2\") pod \"maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:42.149031 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:42.148998    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faede3d1-b97e-4904-b14e-edc6d8bd6d92-maas-api-tls\") pod \"maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:42.151337 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:42.151316    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faede3d1-b97e-4904-b14e-edc6d8bd6d92-maas-api-tls\") pod \"maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:42.320427 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:42.320388    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:42.602986 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:42.602959    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"]
Sep 10 18:04:42.604088 ip-10-0-139-245 kubenswrapper[2549]: W0910 18:04:42.604056    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaede3d1_b97e_4904_b14e_edc6d8bd6d92.slice/crio-9414ca914da1ce0dcdd7dcb6dcabe773e87e9fc13aa0aa7ddb5db524fcfe02d4 WatchSource:0}: Error finding container 9414ca914da1ce0dcdd7dcb6dcabe773e87e9fc13aa0aa7ddb5db524fcfe02d4: Status 404 returned error can't find the container with id 9414ca914da1ce0dcdd7dcb6dcabe773e87e9fc13aa0aa7ddb5db524fcfe02d4
Sep 10 18:04:43.577055 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:43.577020    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj" event={"ID":"faede3d1-b97e-4904-b14e-edc6d8bd6d92","Type":"ContainerStarted","Data":"9f8a7e863ff266cf92022ac2e7a9b4e92df3016a7bcc72b2626fe688a02821ae"}
Sep 10 18:04:43.577241 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:43.577071    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj" event={"ID":"faede3d1-b97e-4904-b14e-edc6d8bd6d92","Type":"ContainerStarted","Data":"9414ca914da1ce0dcdd7dcb6dcabe773e87e9fc13aa0aa7ddb5db524fcfe02d4"}
Sep 10 18:04:43.577241 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:43.577094    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:43.596588 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:43.596465    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj" podStartSLOduration=2.259037823 podStartE2EDuration="2.596447328s" totalImagesPullingTime="337.409505ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:04:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:04:43.596423095 +0000 UTC m=+1408.087474912" watchObservedRunningTime="2026-09-10 18:04:43.596447328 +0000 UTC m=+1408.087499149"
Sep 10 18:04:47.512437 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:47.512393    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:04:49.383231 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.383179    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn" podUID="37ad96e1-92b3-4bce-b554-4f1ec6d48f02" containerName="main" containerID="cri-o://e0ca11e18dde72bfe26eba799ad846bf734d90c48f46b6456587735da4fc8a24" gracePeriod=45
Sep 10 18:04:49.586857 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.586829    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:04:49.606164 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.606113    2549 generic.go:356] "Generic (PLEG): container finished" podID="37ad96e1-92b3-4bce-b554-4f1ec6d48f02" containerID="e0ca11e18dde72bfe26eba799ad846bf734d90c48f46b6456587735da4fc8a24" exitCode=0
Sep 10 18:04:49.606325 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.606188    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn" event={"ID":"37ad96e1-92b3-4bce-b554-4f1ec6d48f02","Type":"ContainerDied","Data":"e0ca11e18dde72bfe26eba799ad846bf734d90c48f46b6456587735da4fc8a24"}
Sep 10 18:04:49.647143 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.647076    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:04:49.726361 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.726322    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-dshm\") pod \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") "
Sep 10 18:04:49.726534 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.726415    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-model-cache\") pod \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") "
Sep 10 18:04:49.726534 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.726460    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-tmp-dir\") pod \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") "
Sep 10 18:04:49.726534 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.726485    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-home\") pod \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") "
Sep 10 18:04:49.726534 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.726510    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-tls-certs\") pod \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") "
Sep 10 18:04:49.726790 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.726549    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xn9lp\" (UniqueName: \"kubernetes.io/projected/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-kube-api-access-xn9lp\") pod \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\" (UID: \"37ad96e1-92b3-4bce-b554-4f1ec6d48f02\") "
Sep 10 18:04:49.726844 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.726747    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-model-cache" (OuterVolumeSpecName: "model-cache") pod "37ad96e1-92b3-4bce-b554-4f1ec6d48f02" (UID: "37ad96e1-92b3-4bce-b554-4f1ec6d48f02"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:04:49.726844 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.726800    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "37ad96e1-92b3-4bce-b554-4f1ec6d48f02" (UID: "37ad96e1-92b3-4bce-b554-4f1ec6d48f02"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:04:49.726844 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.726814    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-home" (OuterVolumeSpecName: "home") pod "37ad96e1-92b3-4bce-b554-4f1ec6d48f02" (UID: "37ad96e1-92b3-4bce-b554-4f1ec6d48f02"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:04:49.726985 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.726969    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-tmp-dir\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:04:49.727045 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.726987    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-home\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:04:49.727045 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.727001    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-model-cache\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:04:49.728579 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.728557    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-dshm" (OuterVolumeSpecName: "dshm") pod "37ad96e1-92b3-4bce-b554-4f1ec6d48f02" (UID: "37ad96e1-92b3-4bce-b554-4f1ec6d48f02"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:04:49.729216 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.729185    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "37ad96e1-92b3-4bce-b554-4f1ec6d48f02" (UID: "37ad96e1-92b3-4bce-b554-4f1ec6d48f02"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:04:49.729216 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.729190    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-kube-api-access-xn9lp" (OuterVolumeSpecName: "kube-api-access-xn9lp") pod "37ad96e1-92b3-4bce-b554-4f1ec6d48f02" (UID: "37ad96e1-92b3-4bce-b554-4f1ec6d48f02"). InnerVolumeSpecName "kube-api-access-xn9lp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:04:49.828047 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.827994    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-tls-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:04:49.828047 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.828035    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xn9lp\" (UniqueName: \"kubernetes.io/projected/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-kube-api-access-xn9lp\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:04:49.828047 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:49.828046    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37ad96e1-92b3-4bce-b554-4f1ec6d48f02-dshm\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:04:50.612546 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:50.612506    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn" event={"ID":"37ad96e1-92b3-4bce-b554-4f1ec6d48f02","Type":"ContainerDied","Data":"be28d1cb1d5bf11c72b49a7d492f593f55b7f101ed10dfa639e8d5f1f8cf0da5"}
Sep 10 18:04:50.612546 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:50.612524    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"
Sep 10 18:04:50.613117 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:50.612554    2549 scope.go:117] "RemoveContainer" containerID="e0ca11e18dde72bfe26eba799ad846bf734d90c48f46b6456587735da4fc8a24"
Sep 10 18:04:50.635791 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:50.635760    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"]
Sep 10 18:04:50.641627 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:50.641603    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6c9ffc/sub-test-model-6c9ffc-kserve-594dc65cdf-b8gnn"]
Sep 10 18:04:52.029260 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:04:52.029228    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37ad96e1-92b3-4bce-b554-4f1ec6d48f02" path="/var/lib/kubelet/pods/37ad96e1-92b3-4bce-b554-4f1ec6d48f02/volumes"
Sep 10 18:05:04.715837 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:04.715800    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:05:23.246209 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.246162    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"]
Sep 10 18:05:23.246637 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.246413    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j" podUID="bd782431-d0e6-44d6-9da3-4e00ecedfbbe" containerName="maas-api" containerID="cri-o://3f40aa7425b7e33e835d7bed4d7007bd2b5cba280fd62dbd8fc659de2807f537" gracePeriod=30
Sep 10 18:05:23.509505 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.509443    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:05:23.650997 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.650960    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlh42\" (UniqueName: \"kubernetes.io/projected/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-kube-api-access-xlh42\") pod \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") "
Sep 10 18:05:23.651171 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.651027    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-metrics-certs\") pod \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") "
Sep 10 18:05:23.651171 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.651071    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-maas-api-tls\") pod \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") "
Sep 10 18:05:23.651273 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.651190    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-openshift-service-ca\") pod \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\" (UID: \"bd782431-d0e6-44d6-9da3-4e00ecedfbbe\") "
Sep 10 18:05:23.651715 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.651637    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd782431-d0e6-44d6-9da3-4e00ecedfbbe" (UID: "bd782431-d0e6-44d6-9da3-4e00ecedfbbe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:05:23.653351 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.653331    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bd782431-d0e6-44d6-9da3-4e00ecedfbbe" (UID: "bd782431-d0e6-44d6-9da3-4e00ecedfbbe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:05:23.653640 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.653614    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-kube-api-access-xlh42" (OuterVolumeSpecName: "kube-api-access-xlh42") pod "bd782431-d0e6-44d6-9da3-4e00ecedfbbe" (UID: "bd782431-d0e6-44d6-9da3-4e00ecedfbbe"). InnerVolumeSpecName "kube-api-access-xlh42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:05:23.653640 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.653617    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd782431-d0e6-44d6-9da3-4e00ecedfbbe" (UID: "bd782431-d0e6-44d6-9da3-4e00ecedfbbe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:05:23.752802 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.752760    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:05:23.752802 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.752794    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlh42\" (UniqueName: \"kubernetes.io/projected/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-kube-api-access-xlh42\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:05:23.752802 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.752805    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:05:23.752802 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.752813    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd782431-d0e6-44d6-9da3-4e00ecedfbbe-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:05:23.771486 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.771401    2549 generic.go:356] "Generic (PLEG): container finished" podID="bd782431-d0e6-44d6-9da3-4e00ecedfbbe" containerID="3f40aa7425b7e33e835d7bed4d7007bd2b5cba280fd62dbd8fc659de2807f537" exitCode=0
Sep 10 18:05:23.771486 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.771452    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"
Sep 10 18:05:23.771641 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.771485    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j" event={"ID":"bd782431-d0e6-44d6-9da3-4e00ecedfbbe","Type":"ContainerDied","Data":"3f40aa7425b7e33e835d7bed4d7007bd2b5cba280fd62dbd8fc659de2807f537"}
Sep 10 18:05:23.771641 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.771537    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j" event={"ID":"bd782431-d0e6-44d6-9da3-4e00ecedfbbe","Type":"ContainerDied","Data":"00c3ce4a225d52638f37acd144135922c9c1ed4e69a2588b459de87333423fa4"}
Sep 10 18:05:23.771641 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.771560    2549 scope.go:117] "RemoveContainer" containerID="3f40aa7425b7e33e835d7bed4d7007bd2b5cba280fd62dbd8fc659de2807f537"
Sep 10 18:05:23.780852 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.780834    2549 scope.go:117] "RemoveContainer" containerID="3f40aa7425b7e33e835d7bed4d7007bd2b5cba280fd62dbd8fc659de2807f537"
Sep 10 18:05:23.781126 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:05:23.781108    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f40aa7425b7e33e835d7bed4d7007bd2b5cba280fd62dbd8fc659de2807f537\": container with ID starting with 3f40aa7425b7e33e835d7bed4d7007bd2b5cba280fd62dbd8fc659de2807f537 not found: ID does not exist" containerID="3f40aa7425b7e33e835d7bed4d7007bd2b5cba280fd62dbd8fc659de2807f537"
Sep 10 18:05:23.781178 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.781132    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f40aa7425b7e33e835d7bed4d7007bd2b5cba280fd62dbd8fc659de2807f537"} err="failed to get container status \"3f40aa7425b7e33e835d7bed4d7007bd2b5cba280fd62dbd8fc659de2807f537\": rpc error: code = NotFound desc = could not find container \"3f40aa7425b7e33e835d7bed4d7007bd2b5cba280fd62dbd8fc659de2807f537\": container with ID starting with 3f40aa7425b7e33e835d7bed4d7007bd2b5cba280fd62dbd8fc659de2807f537 not found: ID does not exist"
Sep 10 18:05:23.805686 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.805651    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"]
Sep 10 18:05:23.811834 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:23.811805    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b38800-6d7799b785-w469j"]
Sep 10 18:05:24.035703 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:24.035608    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd782431-d0e6-44d6-9da3-4e00ecedfbbe" path="/var/lib/kubelet/pods/bd782431-d0e6-44d6-9da3-4e00ecedfbbe/volumes"
Sep 10 18:05:28.432219 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:28.432182    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:05:46.851581 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:46.851542    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"]
Sep 10 18:05:46.852407 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:46.851809    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj" podUID="faede3d1-b97e-4904-b14e-edc6d8bd6d92" containerName="maas-api" containerID="cri-o://9f8a7e863ff266cf92022ac2e7a9b4e92df3016a7bcc72b2626fe688a02821ae" gracePeriod=30
Sep 10 18:05:47.097313 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.097288    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:05:47.159579 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.159488    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faede3d1-b97e-4904-b14e-edc6d8bd6d92-maas-api-tls\") pod \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") "
Sep 10 18:05:47.159579 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.159549    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrws2\" (UniqueName: \"kubernetes.io/projected/faede3d1-b97e-4904-b14e-edc6d8bd6d92-kube-api-access-lrws2\") pod \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") "
Sep 10 18:05:47.159579 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.159570    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faede3d1-b97e-4904-b14e-edc6d8bd6d92-openshift-service-ca\") pod \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") "
Sep 10 18:05:47.159853 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.159635    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faede3d1-b97e-4904-b14e-edc6d8bd6d92-metrics-certs\") pod \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\" (UID: \"faede3d1-b97e-4904-b14e-edc6d8bd6d92\") "
Sep 10 18:05:47.160033 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.159990    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faede3d1-b97e-4904-b14e-edc6d8bd6d92-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "faede3d1-b97e-4904-b14e-edc6d8bd6d92" (UID: "faede3d1-b97e-4904-b14e-edc6d8bd6d92"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:05:47.161727 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.161687    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faede3d1-b97e-4904-b14e-edc6d8bd6d92-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "faede3d1-b97e-4904-b14e-edc6d8bd6d92" (UID: "faede3d1-b97e-4904-b14e-edc6d8bd6d92"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:05:47.161811 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.161727    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faede3d1-b97e-4904-b14e-edc6d8bd6d92-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "faede3d1-b97e-4904-b14e-edc6d8bd6d92" (UID: "faede3d1-b97e-4904-b14e-edc6d8bd6d92"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:05:47.161811 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.161758    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faede3d1-b97e-4904-b14e-edc6d8bd6d92-kube-api-access-lrws2" (OuterVolumeSpecName: "kube-api-access-lrws2") pod "faede3d1-b97e-4904-b14e-edc6d8bd6d92" (UID: "faede3d1-b97e-4904-b14e-edc6d8bd6d92"). InnerVolumeSpecName "kube-api-access-lrws2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:05:47.260492 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.260453    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faede3d1-b97e-4904-b14e-edc6d8bd6d92-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:05:47.260492 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.260489    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrws2\" (UniqueName: \"kubernetes.io/projected/faede3d1-b97e-4904-b14e-edc6d8bd6d92-kube-api-access-lrws2\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:05:47.260492 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.260500    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faede3d1-b97e-4904-b14e-edc6d8bd6d92-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:05:47.260758 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.260509    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faede3d1-b97e-4904-b14e-edc6d8bd6d92-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:05:47.887680 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.887644    2549 generic.go:356] "Generic (PLEG): container finished" podID="faede3d1-b97e-4904-b14e-edc6d8bd6d92" containerID="9f8a7e863ff266cf92022ac2e7a9b4e92df3016a7bcc72b2626fe688a02821ae" exitCode=0
Sep 10 18:05:47.887680 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.887685    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"
Sep 10 18:05:47.888169 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.887726    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj" event={"ID":"faede3d1-b97e-4904-b14e-edc6d8bd6d92","Type":"ContainerDied","Data":"9f8a7e863ff266cf92022ac2e7a9b4e92df3016a7bcc72b2626fe688a02821ae"}
Sep 10 18:05:47.888169 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.887783    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj" event={"ID":"faede3d1-b97e-4904-b14e-edc6d8bd6d92","Type":"ContainerDied","Data":"9414ca914da1ce0dcdd7dcb6dcabe773e87e9fc13aa0aa7ddb5db524fcfe02d4"}
Sep 10 18:05:47.888169 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.887805    2549 scope.go:117] "RemoveContainer" containerID="9f8a7e863ff266cf92022ac2e7a9b4e92df3016a7bcc72b2626fe688a02821ae"
Sep 10 18:05:47.896566 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.896548    2549 scope.go:117] "RemoveContainer" containerID="9f8a7e863ff266cf92022ac2e7a9b4e92df3016a7bcc72b2626fe688a02821ae"
Sep 10 18:05:47.896890 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:05:47.896869    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f8a7e863ff266cf92022ac2e7a9b4e92df3016a7bcc72b2626fe688a02821ae\": container with ID starting with 9f8a7e863ff266cf92022ac2e7a9b4e92df3016a7bcc72b2626fe688a02821ae not found: ID does not exist" containerID="9f8a7e863ff266cf92022ac2e7a9b4e92df3016a7bcc72b2626fe688a02821ae"
Sep 10 18:05:47.896966 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.896898    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f8a7e863ff266cf92022ac2e7a9b4e92df3016a7bcc72b2626fe688a02821ae"} err="failed to get container status \"9f8a7e863ff266cf92022ac2e7a9b4e92df3016a7bcc72b2626fe688a02821ae\": rpc error: code = NotFound desc = could not find container \"9f8a7e863ff266cf92022ac2e7a9b4e92df3016a7bcc72b2626fe688a02821ae\": container with ID starting with 9f8a7e863ff266cf92022ac2e7a9b4e92df3016a7bcc72b2626fe688a02821ae not found: ID does not exist"
Sep 10 18:05:47.912685 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.912661    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"]
Sep 10 18:05:47.915712 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:47.915680    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d439f7-7849f945b4-hnjhj"]
Sep 10 18:05:48.029234 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:48.029195    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faede3d1-b97e-4904-b14e-edc6d8bd6d92" path="/var/lib/kubelet/pods/faede3d1-b97e-4904-b14e-edc6d8bd6d92/volumes"
Sep 10 18:05:51.525576 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:51.525545    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:05:59.532263 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.532230    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"]
Sep 10 18:05:59.532621 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.532591    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd782431-d0e6-44d6-9da3-4e00ecedfbbe" containerName="maas-api"
Sep 10 18:05:59.532621 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.532602    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd782431-d0e6-44d6-9da3-4e00ecedfbbe" containerName="maas-api"
Sep 10 18:05:59.532621 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.532611    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37ad96e1-92b3-4bce-b554-4f1ec6d48f02" containerName="main"
Sep 10 18:05:59.532621 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.532617    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="37ad96e1-92b3-4bce-b554-4f1ec6d48f02" containerName="main"
Sep 10 18:05:59.532773 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.532631    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faede3d1-b97e-4904-b14e-edc6d8bd6d92" containerName="maas-api"
Sep 10 18:05:59.532773 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.532638    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="faede3d1-b97e-4904-b14e-edc6d8bd6d92" containerName="maas-api"
Sep 10 18:05:59.532773 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.532714    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd782431-d0e6-44d6-9da3-4e00ecedfbbe" containerName="maas-api"
Sep 10 18:05:59.532773 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.532726    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="faede3d1-b97e-4904-b14e-edc6d8bd6d92" containerName="maas-api"
Sep 10 18:05:59.532773 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.532734    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="37ad96e1-92b3-4bce-b554-4f1ec6d48f02" containerName="main"
Sep 10 18:05:59.535419 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.535400    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.539241 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.539220    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc23cf\"/\"kube-root-ca.crt\""
Sep 10 18:05:59.540687 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.540623    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc23cf\"/\"rate-test-model-fc23cf-kserve-self-signed-certs\""
Sep 10 18:05:59.540687 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.540639    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc23cf\"/\"default-dockercfg-svv2h\""
Sep 10 18:05:59.540845 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.540678    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-fc23cf\"/\"openshift-service-ca.crt\""
Sep 10 18:05:59.550505 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.550481    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"]
Sep 10 18:05:59.580270 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.580240    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv22m\" (UniqueName: \"kubernetes.io/projected/0c82062f-a787-4bed-b02a-1cde3b204c12-kube-api-access-pv22m\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.580486 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.580464    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-dshm\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.580570 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.580538    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c82062f-a787-4bed-b02a-1cde3b204c12-tls-certs\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.580614 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.580588    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-model-cache\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.580717 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.580683    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-tmp-dir\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.580889 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.580735    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-home\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.681821 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.681777    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c82062f-a787-4bed-b02a-1cde3b204c12-tls-certs\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.681821 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.681824    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-model-cache\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.682084 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.681872    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-tmp-dir\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.682084 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.681893    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-home\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.682084 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.681942    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv22m\" (UniqueName: \"kubernetes.io/projected/0c82062f-a787-4bed-b02a-1cde3b204c12-kube-api-access-pv22m\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.682084 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.681960    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-dshm\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.682382 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.682359    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-model-cache\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.682447 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.682411    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-home\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.682447 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.682432    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-tmp-dir\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.684160 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.684138    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-dshm\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.684383 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.684364    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c82062f-a787-4bed-b02a-1cde3b204c12-tls-certs\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.697495 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.697469    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv22m\" (UniqueName: \"kubernetes.io/projected/0c82062f-a787-4bed-b02a-1cde3b204c12-kube-api-access-pv22m\") pod \"rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") " pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:05:59.844219 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:05:59.844135    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:06:00.115296 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:00.115270    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"]
Sep 10 18:06:00.116894 ip-10-0-139-245 kubenswrapper[2549]: W0910 18:06:00.116868    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c82062f_a787_4bed_b02a_1cde3b204c12.slice/crio-b694bb37d56cd1ff4510a00406a92b9aec0f5dddf19d674684a1b8ed2d96b3dc WatchSource:0}: Error finding container b694bb37d56cd1ff4510a00406a92b9aec0f5dddf19d674684a1b8ed2d96b3dc: Status 404 returned error can't find the container with id b694bb37d56cd1ff4510a00406a92b9aec0f5dddf19d674684a1b8ed2d96b3dc
Sep 10 18:06:00.948060 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:00.948022    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk" event={"ID":"0c82062f-a787-4bed-b02a-1cde3b204c12","Type":"ContainerStarted","Data":"b34109bfb419c2eb5855dd610af29c12b6cf5edfcceead6ea249f1008dcb2332"}
Sep 10 18:06:00.948060 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:00.948065    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk" event={"ID":"0c82062f-a787-4bed-b02a-1cde3b204c12","Type":"ContainerStarted","Data":"b694bb37d56cd1ff4510a00406a92b9aec0f5dddf19d674684a1b8ed2d96b3dc"}
Sep 10 18:06:00.966342 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:00.966283    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk" podStartSLOduration=1.966262794 podStartE2EDuration="1.966262794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:05:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:06:00.965076976 +0000 UTC m=+1485.456128783" watchObservedRunningTime="2026-09-10 18:06:00.966262794 +0000 UTC m=+1485.457314636"
Sep 10 18:06:05.210347 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.210312    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"]
Sep 10 18:06:05.210896 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.210566    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997" podUID="a5679e79-95ee-47ef-8254-2a03514661dd" containerName="maas-api" containerID="cri-o://9c452bc0a0cad20957ebbe8e6e244d21fcd754ffcc36d954f724ed23fb356a04" gracePeriod=30
Sep 10 18:06:05.450641 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.450615    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 18:06:05.540744 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.540636    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8c6kx\" (UniqueName: \"kubernetes.io/projected/a5679e79-95ee-47ef-8254-2a03514661dd-kube-api-access-8c6kx\") pod \"a5679e79-95ee-47ef-8254-2a03514661dd\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") "
Sep 10 18:06:05.540744 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.540674    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5679e79-95ee-47ef-8254-2a03514661dd-maas-api-tls\") pod \"a5679e79-95ee-47ef-8254-2a03514661dd\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") "
Sep 10 18:06:05.540744 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.540727    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5679e79-95ee-47ef-8254-2a03514661dd-openshift-service-ca\") pod \"a5679e79-95ee-47ef-8254-2a03514661dd\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") "
Sep 10 18:06:05.541046 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.540842    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5679e79-95ee-47ef-8254-2a03514661dd-metrics-certs\") pod \"a5679e79-95ee-47ef-8254-2a03514661dd\" (UID: \"a5679e79-95ee-47ef-8254-2a03514661dd\") "
Sep 10 18:06:05.541106 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.541072    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5679e79-95ee-47ef-8254-2a03514661dd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a5679e79-95ee-47ef-8254-2a03514661dd" (UID: "a5679e79-95ee-47ef-8254-2a03514661dd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:06:05.542934 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.542904    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5679e79-95ee-47ef-8254-2a03514661dd-kube-api-access-8c6kx" (OuterVolumeSpecName: "kube-api-access-8c6kx") pod "a5679e79-95ee-47ef-8254-2a03514661dd" (UID: "a5679e79-95ee-47ef-8254-2a03514661dd"). InnerVolumeSpecName "kube-api-access-8c6kx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:06:05.543057 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.542947    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5679e79-95ee-47ef-8254-2a03514661dd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a5679e79-95ee-47ef-8254-2a03514661dd" (UID: "a5679e79-95ee-47ef-8254-2a03514661dd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:06:05.543057 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.542989    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5679e79-95ee-47ef-8254-2a03514661dd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a5679e79-95ee-47ef-8254-2a03514661dd" (UID: "a5679e79-95ee-47ef-8254-2a03514661dd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:06:05.641806 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.641762    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5679e79-95ee-47ef-8254-2a03514661dd-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:06:05.641806 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.641799    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8c6kx\" (UniqueName: \"kubernetes.io/projected/a5679e79-95ee-47ef-8254-2a03514661dd-kube-api-access-8c6kx\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:06:05.641806 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.641809    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5679e79-95ee-47ef-8254-2a03514661dd-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:06:05.641806 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.641818    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5679e79-95ee-47ef-8254-2a03514661dd-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:06:05.972343 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.972280    2549 generic.go:356] "Generic (PLEG): container finished" podID="a5679e79-95ee-47ef-8254-2a03514661dd" containerID="9c452bc0a0cad20957ebbe8e6e244d21fcd754ffcc36d954f724ed23fb356a04" exitCode=0
Sep 10 18:06:05.972510 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.972390    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997" event={"ID":"a5679e79-95ee-47ef-8254-2a03514661dd","Type":"ContainerDied","Data":"9c452bc0a0cad20957ebbe8e6e244d21fcd754ffcc36d954f724ed23fb356a04"}
Sep 10 18:06:05.972510 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.972411    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"
Sep 10 18:06:05.972510 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.972434    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997" event={"ID":"a5679e79-95ee-47ef-8254-2a03514661dd","Type":"ContainerDied","Data":"19338ab841fbad2d9a6105db8d1fbf726f78921a63ab1d1c3104f1def3277e36"}
Sep 10 18:06:05.972510 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.972458    2549 scope.go:117] "RemoveContainer" containerID="9c452bc0a0cad20957ebbe8e6e244d21fcd754ffcc36d954f724ed23fb356a04"
Sep 10 18:06:05.981636 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.981614    2549 scope.go:117] "RemoveContainer" containerID="9c452bc0a0cad20957ebbe8e6e244d21fcd754ffcc36d954f724ed23fb356a04"
Sep 10 18:06:05.981982 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:06:05.981937    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c452bc0a0cad20957ebbe8e6e244d21fcd754ffcc36d954f724ed23fb356a04\": container with ID starting with 9c452bc0a0cad20957ebbe8e6e244d21fcd754ffcc36d954f724ed23fb356a04 not found: ID does not exist" containerID="9c452bc0a0cad20957ebbe8e6e244d21fcd754ffcc36d954f724ed23fb356a04"
Sep 10 18:06:05.981982 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.981966    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c452bc0a0cad20957ebbe8e6e244d21fcd754ffcc36d954f724ed23fb356a04"} err="failed to get container status \"9c452bc0a0cad20957ebbe8e6e244d21fcd754ffcc36d954f724ed23fb356a04\": rpc error: code = NotFound desc = could not find container \"9c452bc0a0cad20957ebbe8e6e244d21fcd754ffcc36d954f724ed23fb356a04\": container with ID starting with 9c452bc0a0cad20957ebbe8e6e244d21fcd754ffcc36d954f724ed23fb356a04 not found: ID does not exist"
Sep 10 18:06:05.994598 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.994566    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"]
Sep 10 18:06:05.999485 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:05.999460    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-971a94-5c46548884-xp997"]
Sep 10 18:06:06.028868 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:06.028838    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5679e79-95ee-47ef-8254-2a03514661dd" path="/var/lib/kubelet/pods/a5679e79-95ee-47ef-8254-2a03514661dd/volumes"
Sep 10 18:06:08.608749 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:08.608701    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:06:09.844487 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:09.844445    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:06:09.844896 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:09.844501    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:06:09.856942 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:09.856916    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:06:10.004472 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:10.004439    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:06:23.216779 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.216744    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"]
Sep 10 18:06:23.217200 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.217001    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz" podUID="ad87b84f-a336-4503-84d4-fd1761b65c4b" containerName="maas-api" containerID="cri-o://c617b88b3def5c8fd601711710da4e4daf8ac5633579b4a9f398b669a737ce9b" gracePeriod=30
Sep 10 18:06:23.352625 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.352578    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz" podUID="ad87b84f-a336-4503-84d4-fd1761b65c4b" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.46:8443/health\": dial tcp 10.134.0.46:8443: connect: connection refused"
Sep 10 18:06:23.461150 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.461119    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:06:23.510223 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.510137    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwcml\" (UniqueName: \"kubernetes.io/projected/ad87b84f-a336-4503-84d4-fd1761b65c4b-kube-api-access-qwcml\") pod \"ad87b84f-a336-4503-84d4-fd1761b65c4b\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") "
Sep 10 18:06:23.510223 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.510209    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-metrics-certs\") pod \"ad87b84f-a336-4503-84d4-fd1761b65c4b\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") "
Sep 10 18:06:23.510426 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.510231    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad87b84f-a336-4503-84d4-fd1761b65c4b-openshift-service-ca\") pod \"ad87b84f-a336-4503-84d4-fd1761b65c4b\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") "
Sep 10 18:06:23.510426 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.510268    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-maas-api-tls\") pod \"ad87b84f-a336-4503-84d4-fd1761b65c4b\" (UID: \"ad87b84f-a336-4503-84d4-fd1761b65c4b\") "
Sep 10 18:06:23.510651 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.510624    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad87b84f-a336-4503-84d4-fd1761b65c4b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ad87b84f-a336-4503-84d4-fd1761b65c4b" (UID: "ad87b84f-a336-4503-84d4-fd1761b65c4b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:06:23.512331 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.512294    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ad87b84f-a336-4503-84d4-fd1761b65c4b" (UID: "ad87b84f-a336-4503-84d4-fd1761b65c4b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:06:23.512455 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.512348    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ad87b84f-a336-4503-84d4-fd1761b65c4b" (UID: "ad87b84f-a336-4503-84d4-fd1761b65c4b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:06:23.512455 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.512380    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad87b84f-a336-4503-84d4-fd1761b65c4b-kube-api-access-qwcml" (OuterVolumeSpecName: "kube-api-access-qwcml") pod "ad87b84f-a336-4503-84d4-fd1761b65c4b" (UID: "ad87b84f-a336-4503-84d4-fd1761b65c4b"). InnerVolumeSpecName "kube-api-access-qwcml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:06:23.611069 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.611031    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:06:23.611069 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.611068    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad87b84f-a336-4503-84d4-fd1761b65c4b-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:06:23.611282 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.611084    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad87b84f-a336-4503-84d4-fd1761b65c4b-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:06:23.611282 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:23.611096    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwcml\" (UniqueName: \"kubernetes.io/projected/ad87b84f-a336-4503-84d4-fd1761b65c4b-kube-api-access-qwcml\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:06:24.060687 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:24.060657    2549 generic.go:356] "Generic (PLEG): container finished" podID="ad87b84f-a336-4503-84d4-fd1761b65c4b" containerID="c617b88b3def5c8fd601711710da4e4daf8ac5633579b4a9f398b669a737ce9b" exitCode=0
Sep 10 18:06:24.060879 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:24.060720    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"
Sep 10 18:06:24.060879 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:24.060740    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz" event={"ID":"ad87b84f-a336-4503-84d4-fd1761b65c4b","Type":"ContainerDied","Data":"c617b88b3def5c8fd601711710da4e4daf8ac5633579b4a9f398b669a737ce9b"}
Sep 10 18:06:24.060879 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:24.060798    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz" event={"ID":"ad87b84f-a336-4503-84d4-fd1761b65c4b","Type":"ContainerDied","Data":"fa0a0c2f56160f8329cc072ae0da87c9b584ae9d86b6ec1c2d308f91adccc903"}
Sep 10 18:06:24.060879 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:24.060827    2549 scope.go:117] "RemoveContainer" containerID="c617b88b3def5c8fd601711710da4e4daf8ac5633579b4a9f398b669a737ce9b"
Sep 10 18:06:24.069356 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:24.069336    2549 scope.go:117] "RemoveContainer" containerID="c617b88b3def5c8fd601711710da4e4daf8ac5633579b4a9f398b669a737ce9b"
Sep 10 18:06:24.069631 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:06:24.069612    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c617b88b3def5c8fd601711710da4e4daf8ac5633579b4a9f398b669a737ce9b\": container with ID starting with c617b88b3def5c8fd601711710da4e4daf8ac5633579b4a9f398b669a737ce9b not found: ID does not exist" containerID="c617b88b3def5c8fd601711710da4e4daf8ac5633579b4a9f398b669a737ce9b"
Sep 10 18:06:24.069716 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:24.069638    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c617b88b3def5c8fd601711710da4e4daf8ac5633579b4a9f398b669a737ce9b"} err="failed to get container status \"c617b88b3def5c8fd601711710da4e4daf8ac5633579b4a9f398b669a737ce9b\": rpc error: code = NotFound desc = could not find container \"c617b88b3def5c8fd601711710da4e4daf8ac5633579b4a9f398b669a737ce9b\": container with ID starting with c617b88b3def5c8fd601711710da4e4daf8ac5633579b4a9f398b669a737ce9b not found: ID does not exist"
Sep 10 18:06:24.081627 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:24.081603    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"]
Sep 10 18:06:24.083144 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:24.083124    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-09fbb5-9c8dbdf7b-cpzmz"]
Sep 10 18:06:24.383361 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:24.383278    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:06:26.029483 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:26.029439    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad87b84f-a336-4503-84d4-fd1761b65c4b" path="/var/lib/kubelet/pods/ad87b84f-a336-4503-84d4-fd1761b65c4b/volumes"
Sep 10 18:06:34.814703 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:34.814666    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:06:40.918753 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:40.918662    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"]
Sep 10 18:06:45.694722 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:45.694670    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:06:46.001214 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.001175    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"]
Sep 10 18:06:46.001751 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.001730    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5679e79-95ee-47ef-8254-2a03514661dd" containerName="maas-api"
Sep 10 18:06:46.001836 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.001753    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5679e79-95ee-47ef-8254-2a03514661dd" containerName="maas-api"
Sep 10 18:06:46.001836 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.001801    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad87b84f-a336-4503-84d4-fd1761b65c4b" containerName="maas-api"
Sep 10 18:06:46.001836 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.001811    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad87b84f-a336-4503-84d4-fd1761b65c4b" containerName="maas-api"
Sep 10 18:06:46.002007 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.001903    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5679e79-95ee-47ef-8254-2a03514661dd" containerName="maas-api"
Sep 10 18:06:46.002007 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.001914    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad87b84f-a336-4503-84d4-fd1761b65c4b" containerName="maas-api"
Sep 10 18:06:46.007022 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.006997    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:46.009753 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.009725    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-732455-serving-cert\""
Sep 10 18:06:46.015796 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.015127    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"]
Sep 10 18:06:46.116828 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.116793    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f28kz\" (UniqueName: \"kubernetes.io/projected/f4327a21-6426-4ea9-9a66-1c2db2957810-kube-api-access-f28kz\") pod \"maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:46.116828 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.116840    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4327a21-6426-4ea9-9a66-1c2db2957810-maas-api-tls\") pod \"maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:46.117067 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.116882    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4327a21-6426-4ea9-9a66-1c2db2957810-metrics-certs\") pod \"maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:46.117067 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.116911    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4327a21-6426-4ea9-9a66-1c2db2957810-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:46.218529 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.218494    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f28kz\" (UniqueName: \"kubernetes.io/projected/f4327a21-6426-4ea9-9a66-1c2db2957810-kube-api-access-f28kz\") pod \"maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:46.218529 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.218532    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4327a21-6426-4ea9-9a66-1c2db2957810-maas-api-tls\") pod \"maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:46.218783 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.218562    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4327a21-6426-4ea9-9a66-1c2db2957810-metrics-certs\") pod \"maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:46.218783 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.218595    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4327a21-6426-4ea9-9a66-1c2db2957810-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:46.219375 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.219347    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4327a21-6426-4ea9-9a66-1c2db2957810-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:46.221122 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.221096    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4327a21-6426-4ea9-9a66-1c2db2957810-maas-api-tls\") pod \"maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:46.221216 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.221101    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4327a21-6426-4ea9-9a66-1c2db2957810-metrics-certs\") pod \"maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:46.227398 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.227380    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f28kz\" (UniqueName: \"kubernetes.io/projected/f4327a21-6426-4ea9-9a66-1c2db2957810-kube-api-access-f28kz\") pod \"maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:46.319197 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.319106    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:46.590463 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:46.590438    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"]
Sep 10 18:06:46.592538 ip-10-0-139-245 kubenswrapper[2549]: W0910 18:06:46.592511    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4327a21_6426_4ea9_9a66_1c2db2957810.slice/crio-e8beb0b6d4bc46f4f5e15c79e87947e9a2f50693c975980e778dd350e3b6da2a WatchSource:0}: Error finding container e8beb0b6d4bc46f4f5e15c79e87947e9a2f50693c975980e778dd350e3b6da2a: Status 404 returned error can't find the container with id e8beb0b6d4bc46f4f5e15c79e87947e9a2f50693c975980e778dd350e3b6da2a
Sep 10 18:06:47.171955 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:47.171920    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc" event={"ID":"f4327a21-6426-4ea9-9a66-1c2db2957810","Type":"ContainerStarted","Data":"5218f8df1820853e4e86d1f366cf2bd5f335b73ff2ea0a353782fedc46c26c67"}
Sep 10 18:06:47.172348 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:47.171976    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc" event={"ID":"f4327a21-6426-4ea9-9a66-1c2db2957810","Type":"ContainerStarted","Data":"e8beb0b6d4bc46f4f5e15c79e87947e9a2f50693c975980e778dd350e3b6da2a"}
Sep 10 18:06:47.172348 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:47.171996    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:47.193495 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:47.193439    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc" podStartSLOduration=1.771567983 podStartE2EDuration="2.193419876s" totalImagesPullingTime="421.851893ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:06:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:06:47.1921394 +0000 UTC m=+1531.683191213" watchObservedRunningTime="2026-09-10 18:06:47.193419876 +0000 UTC m=+1531.684471684"
Sep 10 18:06:53.181446 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:53.181413    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:06:55.938206 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:55.938156    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk" podUID="0c82062f-a787-4bed-b02a-1cde3b204c12" containerName="main" containerID="cri-o://b34109bfb419c2eb5855dd610af29c12b6cf5edfcceead6ea249f1008dcb2332" gracePeriod=45
Sep 10 18:06:56.185967 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.185942    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:06:56.215190 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.215152    2549 generic.go:356] "Generic (PLEG): container finished" podID="0c82062f-a787-4bed-b02a-1cde3b204c12" containerID="b34109bfb419c2eb5855dd610af29c12b6cf5edfcceead6ea249f1008dcb2332" exitCode=0
Sep 10 18:06:56.215382 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.215204    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"
Sep 10 18:06:56.215382 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.215241    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk" event={"ID":"0c82062f-a787-4bed-b02a-1cde3b204c12","Type":"ContainerDied","Data":"b34109bfb419c2eb5855dd610af29c12b6cf5edfcceead6ea249f1008dcb2332"}
Sep 10 18:06:56.215382 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.215298    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk" event={"ID":"0c82062f-a787-4bed-b02a-1cde3b204c12","Type":"ContainerDied","Data":"b694bb37d56cd1ff4510a00406a92b9aec0f5dddf19d674684a1b8ed2d96b3dc"}
Sep 10 18:06:56.215382 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.215322    2549 scope.go:117] "RemoveContainer" containerID="b34109bfb419c2eb5855dd610af29c12b6cf5edfcceead6ea249f1008dcb2332"
Sep 10 18:06:56.224304 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.224284    2549 scope.go:117] "RemoveContainer" containerID="b34109bfb419c2eb5855dd610af29c12b6cf5edfcceead6ea249f1008dcb2332"
Sep 10 18:06:56.224602 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:06:56.224579    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b34109bfb419c2eb5855dd610af29c12b6cf5edfcceead6ea249f1008dcb2332\": container with ID starting with b34109bfb419c2eb5855dd610af29c12b6cf5edfcceead6ea249f1008dcb2332 not found: ID does not exist" containerID="b34109bfb419c2eb5855dd610af29c12b6cf5edfcceead6ea249f1008dcb2332"
Sep 10 18:06:56.224671 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.224612    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b34109bfb419c2eb5855dd610af29c12b6cf5edfcceead6ea249f1008dcb2332"} err="failed to get container status \"b34109bfb419c2eb5855dd610af29c12b6cf5edfcceead6ea249f1008dcb2332\": rpc error: code = NotFound desc = could not find container \"b34109bfb419c2eb5855dd610af29c12b6cf5edfcceead6ea249f1008dcb2332\": container with ID starting with b34109bfb419c2eb5855dd610af29c12b6cf5edfcceead6ea249f1008dcb2332 not found: ID does not exist"
Sep 10 18:06:56.309798 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.309759    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-home\") pod \"0c82062f-a787-4bed-b02a-1cde3b204c12\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") "
Sep 10 18:06:56.309798 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.309799    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-tmp-dir\") pod \"0c82062f-a787-4bed-b02a-1cde3b204c12\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") "
Sep 10 18:06:56.310025 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.309854    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pv22m\" (UniqueName: \"kubernetes.io/projected/0c82062f-a787-4bed-b02a-1cde3b204c12-kube-api-access-pv22m\") pod \"0c82062f-a787-4bed-b02a-1cde3b204c12\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") "
Sep 10 18:06:56.310025 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.309901    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-dshm\") pod \"0c82062f-a787-4bed-b02a-1cde3b204c12\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") "
Sep 10 18:06:56.310025 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.309927    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c82062f-a787-4bed-b02a-1cde3b204c12-tls-certs\") pod \"0c82062f-a787-4bed-b02a-1cde3b204c12\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") "
Sep 10 18:06:56.310025 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.309998    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-model-cache\") pod \"0c82062f-a787-4bed-b02a-1cde3b204c12\" (UID: \"0c82062f-a787-4bed-b02a-1cde3b204c12\") "
Sep 10 18:06:56.310254 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.310060    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0c82062f-a787-4bed-b02a-1cde3b204c12" (UID: "0c82062f-a787-4bed-b02a-1cde3b204c12"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:06:56.310254 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.310076    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-home" (OuterVolumeSpecName: "home") pod "0c82062f-a787-4bed-b02a-1cde3b204c12" (UID: "0c82062f-a787-4bed-b02a-1cde3b204c12"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:06:56.310366 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.310303    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-model-cache" (OuterVolumeSpecName: "model-cache") pod "0c82062f-a787-4bed-b02a-1cde3b204c12" (UID: "0c82062f-a787-4bed-b02a-1cde3b204c12"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:06:56.310366 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.310315    2549 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-home\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:06:56.310366 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.310330    2549 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-tmp-dir\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:06:56.312234 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.312200    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c82062f-a787-4bed-b02a-1cde3b204c12-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0c82062f-a787-4bed-b02a-1cde3b204c12" (UID: "0c82062f-a787-4bed-b02a-1cde3b204c12"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:06:56.312492 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.312470    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-dshm" (OuterVolumeSpecName: "dshm") pod "0c82062f-a787-4bed-b02a-1cde3b204c12" (UID: "0c82062f-a787-4bed-b02a-1cde3b204c12"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:06:56.312563 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.312475    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c82062f-a787-4bed-b02a-1cde3b204c12-kube-api-access-pv22m" (OuterVolumeSpecName: "kube-api-access-pv22m") pod "0c82062f-a787-4bed-b02a-1cde3b204c12" (UID: "0c82062f-a787-4bed-b02a-1cde3b204c12"). InnerVolumeSpecName "kube-api-access-pv22m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:06:56.411030 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.410994    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-model-cache\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:06:56.411030 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.411024    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv22m\" (UniqueName: \"kubernetes.io/projected/0c82062f-a787-4bed-b02a-1cde3b204c12-kube-api-access-pv22m\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:06:56.411030 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.411036    2549 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c82062f-a787-4bed-b02a-1cde3b204c12-dshm\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:06:56.411314 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.411044    2549 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c82062f-a787-4bed-b02a-1cde3b204c12-tls-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:06:56.510971 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.510884    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:06:56.540350 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.540303    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"]
Sep 10 18:06:56.542684 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:56.542656    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-fc23cf/rate-test-model-fc23cf-kserve-86d4d478c5-bg9tk"]
Sep 10 18:06:58.029640 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:06:58.029604    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c82062f-a787-4bed-b02a-1cde3b204c12" path="/var/lib/kubelet/pods/0c82062f-a787-4bed-b02a-1cde3b204c12/volumes"
Sep 10 18:07:15.052478 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.052438    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"]
Sep 10 18:07:15.052945 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.052845    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c82062f-a787-4bed-b02a-1cde3b204c12" containerName="main"
Sep 10 18:07:15.052945 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.052857    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c82062f-a787-4bed-b02a-1cde3b204c12" containerName="main"
Sep 10 18:07:15.052945 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.052925    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c82062f-a787-4bed-b02a-1cde3b204c12" containerName="main"
Sep 10 18:07:15.057420 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.057400    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:15.060316 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.060293    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-4d9405-serving-cert\""
Sep 10 18:07:15.074120 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.074086    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"]
Sep 10 18:07:15.079830 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.079799    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63086064-884c-4f33-bf02-298f0bf1d77e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:15.080013 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.079837    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqq6z\" (UniqueName: \"kubernetes.io/projected/63086064-884c-4f33-bf02-298f0bf1d77e-kube-api-access-wqq6z\") pod \"maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:15.080013 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.079858    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63086064-884c-4f33-bf02-298f0bf1d77e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:15.080013 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.079887    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63086064-884c-4f33-bf02-298f0bf1d77e-metrics-certs\") pod \"maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:15.180712 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.180657    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63086064-884c-4f33-bf02-298f0bf1d77e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:15.180712 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.180718    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqq6z\" (UniqueName: \"kubernetes.io/projected/63086064-884c-4f33-bf02-298f0bf1d77e-kube-api-access-wqq6z\") pod \"maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:15.180929 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.180737    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63086064-884c-4f33-bf02-298f0bf1d77e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:15.180929 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.180752    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63086064-884c-4f33-bf02-298f0bf1d77e-metrics-certs\") pod \"maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:15.181391 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.181356    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63086064-884c-4f33-bf02-298f0bf1d77e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:15.183151 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.183118    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63086064-884c-4f33-bf02-298f0bf1d77e-metrics-certs\") pod \"maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:15.183286 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.183207    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63086064-884c-4f33-bf02-298f0bf1d77e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:15.203330 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.203304    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqq6z\" (UniqueName: \"kubernetes.io/projected/63086064-884c-4f33-bf02-298f0bf1d77e-kube-api-access-wqq6z\") pod \"maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:15.368171 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.368085    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:15.649230 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:15.649155    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"]
Sep 10 18:07:15.652587 ip-10-0-139-245 kubenswrapper[2549]: W0910 18:07:15.652556    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63086064_884c_4f33_bf02_298f0bf1d77e.slice/crio-d8b7e6a5435f95b897b15014f6d3a1ab5bee76031ecc1ecc2e3979fd7ba034da WatchSource:0}: Error finding container d8b7e6a5435f95b897b15014f6d3a1ab5bee76031ecc1ecc2e3979fd7ba034da: Status 404 returned error can't find the container with id d8b7e6a5435f95b897b15014f6d3a1ab5bee76031ecc1ecc2e3979fd7ba034da
Sep 10 18:07:16.316113 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:16.316020    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn" event={"ID":"63086064-884c-4f33-bf02-298f0bf1d77e","Type":"ContainerStarted","Data":"c7706c55f1273898a6dedf74ab2eb0d49ec40bec0ad44874498e387d1e0479c0"}
Sep 10 18:07:16.316113 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:16.316077    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn" event={"ID":"63086064-884c-4f33-bf02-298f0bf1d77e","Type":"ContainerStarted","Data":"d8b7e6a5435f95b897b15014f6d3a1ab5bee76031ecc1ecc2e3979fd7ba034da"}
Sep 10 18:07:16.316113 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:16.316106    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:16.337129 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:16.337083    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn" podStartSLOduration=0.968121385 podStartE2EDuration="1.337067792s" totalImagesPullingTime="368.946407ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:07:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:07:16.336968393 +0000 UTC m=+1560.828020202" watchObservedRunningTime="2026-09-10 18:07:16.337067792 +0000 UTC m=+1560.828119599"
Sep 10 18:07:22.325833 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:22.325802    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:07:22.417432 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:07:22.417400    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:08:03.103006 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:03.102963    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:08:14.099054 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:14.098972    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:08:24.510252 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.510217    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"]
Sep 10 18:08:24.513646 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.513628    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:24.516353 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.516331    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-3b8881-serving-cert\""
Sep 10 18:08:24.524849 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.524827    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"]
Sep 10 18:08:24.628663 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.628623    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb1865e7-7700-4e3c-9c0d-bd5334198502-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:24.628897 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.628675    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb1865e7-7700-4e3c-9c0d-bd5334198502-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:24.628897 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.628729    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb1865e7-7700-4e3c-9c0d-bd5334198502-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:24.628897 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.628809    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgwll\" (UniqueName: \"kubernetes.io/projected/fb1865e7-7700-4e3c-9c0d-bd5334198502-kube-api-access-rgwll\") pod \"maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:24.730086 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.730048    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb1865e7-7700-4e3c-9c0d-bd5334198502-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:24.730315 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.730100    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb1865e7-7700-4e3c-9c0d-bd5334198502-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:24.730315 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.730138    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb1865e7-7700-4e3c-9c0d-bd5334198502-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:24.730315 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.730201    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgwll\" (UniqueName: \"kubernetes.io/projected/fb1865e7-7700-4e3c-9c0d-bd5334198502-kube-api-access-rgwll\") pod \"maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:24.730857 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.730834    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb1865e7-7700-4e3c-9c0d-bd5334198502-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:24.732570 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.732548    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb1865e7-7700-4e3c-9c0d-bd5334198502-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:24.732657 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.732588    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb1865e7-7700-4e3c-9c0d-bd5334198502-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:24.741023 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.740999    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgwll\" (UniqueName: \"kubernetes.io/projected/fb1865e7-7700-4e3c-9c0d-bd5334198502-kube-api-access-rgwll\") pod \"maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:24.824719 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:24.824581    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:25.127926 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:25.127891    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"]
Sep 10 18:08:25.129647 ip-10-0-139-245 kubenswrapper[2549]: W0910 18:08:25.129621    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb1865e7_7700_4e3c_9c0d_bd5334198502.slice/crio-e266677fe112eabbfe8dc8709be45198e773266cfde1339b6e11c09fe96c2c9d WatchSource:0}: Error finding container e266677fe112eabbfe8dc8709be45198e773266cfde1339b6e11c09fe96c2c9d: Status 404 returned error can't find the container with id e266677fe112eabbfe8dc8709be45198e773266cfde1339b6e11c09fe96c2c9d
Sep 10 18:08:25.642534 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:25.642500    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc" event={"ID":"fb1865e7-7700-4e3c-9c0d-bd5334198502","Type":"ContainerStarted","Data":"7f6507d7843879b8e4d4715468e07dace5b36617bb59173b1390229572bcb044"}
Sep 10 18:08:25.642993 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:25.642552    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc" event={"ID":"fb1865e7-7700-4e3c-9c0d-bd5334198502","Type":"ContainerStarted","Data":"e266677fe112eabbfe8dc8709be45198e773266cfde1339b6e11c09fe96c2c9d"}
Sep 10 18:08:25.642993 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:25.642605    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:25.668577 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:25.668527    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc" podStartSLOduration=1.3197124009999999 podStartE2EDuration="1.668489892s" totalImagesPullingTime="348.777491ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:08:25.666106245 +0000 UTC m=+1630.157158054" watchObservedRunningTime="2026-09-10 18:08:25.668489892 +0000 UTC m=+1630.159541700"
Sep 10 18:08:31.652494 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:31.652458    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:33.712209 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:33.712176    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:08:55.139071 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.139036    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"]
Sep 10 18:08:55.139531 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.139252    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc" podUID="fb1865e7-7700-4e3c-9c0d-bd5334198502" containerName="maas-api" containerID="cri-o://7f6507d7843879b8e4d4715468e07dace5b36617bb59173b1390229572bcb044" gracePeriod=30
Sep 10 18:08:55.384976 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.384950    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:55.519609 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.519576    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb1865e7-7700-4e3c-9c0d-bd5334198502-openshift-service-ca\") pod \"fb1865e7-7700-4e3c-9c0d-bd5334198502\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") "
Sep 10 18:08:55.519794 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.519643    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb1865e7-7700-4e3c-9c0d-bd5334198502-maas-api-tls\") pod \"fb1865e7-7700-4e3c-9c0d-bd5334198502\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") "
Sep 10 18:08:55.519794 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.519687    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb1865e7-7700-4e3c-9c0d-bd5334198502-metrics-certs\") pod \"fb1865e7-7700-4e3c-9c0d-bd5334198502\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") "
Sep 10 18:08:55.519794 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.519726    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgwll\" (UniqueName: \"kubernetes.io/projected/fb1865e7-7700-4e3c-9c0d-bd5334198502-kube-api-access-rgwll\") pod \"fb1865e7-7700-4e3c-9c0d-bd5334198502\" (UID: \"fb1865e7-7700-4e3c-9c0d-bd5334198502\") "
Sep 10 18:08:55.520032 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.519996    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb1865e7-7700-4e3c-9c0d-bd5334198502-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fb1865e7-7700-4e3c-9c0d-bd5334198502" (UID: "fb1865e7-7700-4e3c-9c0d-bd5334198502"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:08:55.521796 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.521764    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1865e7-7700-4e3c-9c0d-bd5334198502-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fb1865e7-7700-4e3c-9c0d-bd5334198502" (UID: "fb1865e7-7700-4e3c-9c0d-bd5334198502"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:08:55.521796 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.521781    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1865e7-7700-4e3c-9c0d-bd5334198502-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fb1865e7-7700-4e3c-9c0d-bd5334198502" (UID: "fb1865e7-7700-4e3c-9c0d-bd5334198502"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:08:55.521939 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.521806    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb1865e7-7700-4e3c-9c0d-bd5334198502-kube-api-access-rgwll" (OuterVolumeSpecName: "kube-api-access-rgwll") pod "fb1865e7-7700-4e3c-9c0d-bd5334198502" (UID: "fb1865e7-7700-4e3c-9c0d-bd5334198502"). InnerVolumeSpecName "kube-api-access-rgwll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:08:55.621223 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.621180    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb1865e7-7700-4e3c-9c0d-bd5334198502-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:08:55.621223 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.621215    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb1865e7-7700-4e3c-9c0d-bd5334198502-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:08:55.621223 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.621228    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgwll\" (UniqueName: \"kubernetes.io/projected/fb1865e7-7700-4e3c-9c0d-bd5334198502-kube-api-access-rgwll\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:08:55.621463 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.621242    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb1865e7-7700-4e3c-9c0d-bd5334198502-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:08:55.783632 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.783537    2549 generic.go:356] "Generic (PLEG): container finished" podID="fb1865e7-7700-4e3c-9c0d-bd5334198502" containerID="7f6507d7843879b8e4d4715468e07dace5b36617bb59173b1390229572bcb044" exitCode=0
Sep 10 18:08:55.783632 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.783588    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"
Sep 10 18:08:55.783632 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.783620    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc" event={"ID":"fb1865e7-7700-4e3c-9c0d-bd5334198502","Type":"ContainerDied","Data":"7f6507d7843879b8e4d4715468e07dace5b36617bb59173b1390229572bcb044"}
Sep 10 18:08:55.783947 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.783675    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc" event={"ID":"fb1865e7-7700-4e3c-9c0d-bd5334198502","Type":"ContainerDied","Data":"e266677fe112eabbfe8dc8709be45198e773266cfde1339b6e11c09fe96c2c9d"}
Sep 10 18:08:55.783947 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.783717    2549 scope.go:117] "RemoveContainer" containerID="7f6507d7843879b8e4d4715468e07dace5b36617bb59173b1390229572bcb044"
Sep 10 18:08:55.793264 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.793247    2549 scope.go:117] "RemoveContainer" containerID="7f6507d7843879b8e4d4715468e07dace5b36617bb59173b1390229572bcb044"
Sep 10 18:08:55.793571 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:08:55.793551    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f6507d7843879b8e4d4715468e07dace5b36617bb59173b1390229572bcb044\": container with ID starting with 7f6507d7843879b8e4d4715468e07dace5b36617bb59173b1390229572bcb044 not found: ID does not exist" containerID="7f6507d7843879b8e4d4715468e07dace5b36617bb59173b1390229572bcb044"
Sep 10 18:08:55.793621 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.793580    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f6507d7843879b8e4d4715468e07dace5b36617bb59173b1390229572bcb044"} err="failed to get container status \"7f6507d7843879b8e4d4715468e07dace5b36617bb59173b1390229572bcb044\": rpc error: code = NotFound desc = could not find container \"7f6507d7843879b8e4d4715468e07dace5b36617bb59173b1390229572bcb044\": container with ID starting with 7f6507d7843879b8e4d4715468e07dace5b36617bb59173b1390229572bcb044 not found: ID does not exist"
Sep 10 18:08:55.804823 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.804796    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"]
Sep 10 18:08:55.810826 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:55.810803    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b8881-55d99f7f5d-fcfkc"]
Sep 10 18:08:56.030676 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:56.030642    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb1865e7-7700-4e3c-9c0d-bd5334198502" path="/var/lib/kubelet/pods/fb1865e7-7700-4e3c-9c0d-bd5334198502/volumes"
Sep 10 18:08:59.903186 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:08:59.903150    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:09:18.797663 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:18.797621    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:09:21.052133 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.052092    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"]
Sep 10 18:09:21.052620 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.052359    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc" podUID="f4327a21-6426-4ea9-9a66-1c2db2957810" containerName="maas-api" containerID="cri-o://5218f8df1820853e4e86d1f366cf2bd5f335b73ff2ea0a353782fedc46c26c67" gracePeriod=30
Sep 10 18:09:21.306258 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.306184    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:09:21.355837 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.355799    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4327a21-6426-4ea9-9a66-1c2db2957810-openshift-service-ca\") pod \"f4327a21-6426-4ea9-9a66-1c2db2957810\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") "
Sep 10 18:09:21.355837 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.355842    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4327a21-6426-4ea9-9a66-1c2db2957810-metrics-certs\") pod \"f4327a21-6426-4ea9-9a66-1c2db2957810\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") "
Sep 10 18:09:21.356119 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.355904    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4327a21-6426-4ea9-9a66-1c2db2957810-maas-api-tls\") pod \"f4327a21-6426-4ea9-9a66-1c2db2957810\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") "
Sep 10 18:09:21.356119 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.355940    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f28kz\" (UniqueName: \"kubernetes.io/projected/f4327a21-6426-4ea9-9a66-1c2db2957810-kube-api-access-f28kz\") pod \"f4327a21-6426-4ea9-9a66-1c2db2957810\" (UID: \"f4327a21-6426-4ea9-9a66-1c2db2957810\") "
Sep 10 18:09:21.356243 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.356220    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4327a21-6426-4ea9-9a66-1c2db2957810-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f4327a21-6426-4ea9-9a66-1c2db2957810" (UID: "f4327a21-6426-4ea9-9a66-1c2db2957810"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:09:21.358196 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.358156    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4327a21-6426-4ea9-9a66-1c2db2957810-kube-api-access-f28kz" (OuterVolumeSpecName: "kube-api-access-f28kz") pod "f4327a21-6426-4ea9-9a66-1c2db2957810" (UID: "f4327a21-6426-4ea9-9a66-1c2db2957810"). InnerVolumeSpecName "kube-api-access-f28kz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:09:21.358520 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.358498    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4327a21-6426-4ea9-9a66-1c2db2957810-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f4327a21-6426-4ea9-9a66-1c2db2957810" (UID: "f4327a21-6426-4ea9-9a66-1c2db2957810"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:09:21.358582 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.358518    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4327a21-6426-4ea9-9a66-1c2db2957810-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f4327a21-6426-4ea9-9a66-1c2db2957810" (UID: "f4327a21-6426-4ea9-9a66-1c2db2957810"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:09:21.456636 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.456584    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4327a21-6426-4ea9-9a66-1c2db2957810-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:09:21.456636 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.456624    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4327a21-6426-4ea9-9a66-1c2db2957810-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:09:21.456636 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.456639    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4327a21-6426-4ea9-9a66-1c2db2957810-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:09:21.456636 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.456649    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f28kz\" (UniqueName: \"kubernetes.io/projected/f4327a21-6426-4ea9-9a66-1c2db2957810-kube-api-access-f28kz\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:09:21.915138 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.915101    2549 generic.go:356] "Generic (PLEG): container finished" podID="f4327a21-6426-4ea9-9a66-1c2db2957810" containerID="5218f8df1820853e4e86d1f366cf2bd5f335b73ff2ea0a353782fedc46c26c67" exitCode=0
Sep 10 18:09:21.915341 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.915149    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"
Sep 10 18:09:21.915341 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.915177    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc" event={"ID":"f4327a21-6426-4ea9-9a66-1c2db2957810","Type":"ContainerDied","Data":"5218f8df1820853e4e86d1f366cf2bd5f335b73ff2ea0a353782fedc46c26c67"}
Sep 10 18:09:21.915341 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.915229    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc" event={"ID":"f4327a21-6426-4ea9-9a66-1c2db2957810","Type":"ContainerDied","Data":"e8beb0b6d4bc46f4f5e15c79e87947e9a2f50693c975980e778dd350e3b6da2a"}
Sep 10 18:09:21.915341 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.915251    2549 scope.go:117] "RemoveContainer" containerID="5218f8df1820853e4e86d1f366cf2bd5f335b73ff2ea0a353782fedc46c26c67"
Sep 10 18:09:21.931410 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.931387    2549 scope.go:117] "RemoveContainer" containerID="5218f8df1820853e4e86d1f366cf2bd5f335b73ff2ea0a353782fedc46c26c67"
Sep 10 18:09:21.931739 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:09:21.931718    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5218f8df1820853e4e86d1f366cf2bd5f335b73ff2ea0a353782fedc46c26c67\": container with ID starting with 5218f8df1820853e4e86d1f366cf2bd5f335b73ff2ea0a353782fedc46c26c67 not found: ID does not exist" containerID="5218f8df1820853e4e86d1f366cf2bd5f335b73ff2ea0a353782fedc46c26c67"
Sep 10 18:09:21.931794 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.931747    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5218f8df1820853e4e86d1f366cf2bd5f335b73ff2ea0a353782fedc46c26c67"} err="failed to get container status \"5218f8df1820853e4e86d1f366cf2bd5f335b73ff2ea0a353782fedc46c26c67\": rpc error: code = NotFound desc = could not find container \"5218f8df1820853e4e86d1f366cf2bd5f335b73ff2ea0a353782fedc46c26c67\": container with ID starting with 5218f8df1820853e4e86d1f366cf2bd5f335b73ff2ea0a353782fedc46c26c67 not found: ID does not exist"
Sep 10 18:09:21.944164 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.944133    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"]
Sep 10 18:09:21.946793 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:21.946771    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-732455-ddc4b7597-wgrcc"]
Sep 10 18:09:22.029907 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:22.029876    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4327a21-6426-4ea9-9a66-1c2db2957810" path="/var/lib/kubelet/pods/f4327a21-6426-4ea9-9a66-1c2db2957810/volumes"
Sep 10 18:09:31.610229 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:09:31.610190    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:10:20.684914 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:20.684875    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"]
Sep 10 18:10:20.685430 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:20.685327    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn" podUID="63086064-884c-4f33-bf02-298f0bf1d77e" containerName="maas-api" containerID="cri-o://c7706c55f1273898a6dedf74ab2eb0d49ec40bec0ad44874498e387d1e0479c0" gracePeriod=30
Sep 10 18:10:20.941971 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:20.941896    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:10:21.025803 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.025769    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63086064-884c-4f33-bf02-298f0bf1d77e-maas-api-tls\") pod \"63086064-884c-4f33-bf02-298f0bf1d77e\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") "
Sep 10 18:10:21.026065 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.025818    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63086064-884c-4f33-bf02-298f0bf1d77e-metrics-certs\") pod \"63086064-884c-4f33-bf02-298f0bf1d77e\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") "
Sep 10 18:10:21.026065 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.025847    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63086064-884c-4f33-bf02-298f0bf1d77e-openshift-service-ca\") pod \"63086064-884c-4f33-bf02-298f0bf1d77e\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") "
Sep 10 18:10:21.026065 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.025917    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqq6z\" (UniqueName: \"kubernetes.io/projected/63086064-884c-4f33-bf02-298f0bf1d77e-kube-api-access-wqq6z\") pod \"63086064-884c-4f33-bf02-298f0bf1d77e\" (UID: \"63086064-884c-4f33-bf02-298f0bf1d77e\") "
Sep 10 18:10:21.026258 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.026233    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63086064-884c-4f33-bf02-298f0bf1d77e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "63086064-884c-4f33-bf02-298f0bf1d77e" (UID: "63086064-884c-4f33-bf02-298f0bf1d77e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:10:21.026370 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.026351    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63086064-884c-4f33-bf02-298f0bf1d77e-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:10:21.028086 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.028057    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63086064-884c-4f33-bf02-298f0bf1d77e-kube-api-access-wqq6z" (OuterVolumeSpecName: "kube-api-access-wqq6z") pod "63086064-884c-4f33-bf02-298f0bf1d77e" (UID: "63086064-884c-4f33-bf02-298f0bf1d77e"). InnerVolumeSpecName "kube-api-access-wqq6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:10:21.028218 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.028171    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63086064-884c-4f33-bf02-298f0bf1d77e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "63086064-884c-4f33-bf02-298f0bf1d77e" (UID: "63086064-884c-4f33-bf02-298f0bf1d77e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:10:21.028381 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.028359    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63086064-884c-4f33-bf02-298f0bf1d77e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "63086064-884c-4f33-bf02-298f0bf1d77e" (UID: "63086064-884c-4f33-bf02-298f0bf1d77e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:10:21.127723 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.127650    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqq6z\" (UniqueName: \"kubernetes.io/projected/63086064-884c-4f33-bf02-298f0bf1d77e-kube-api-access-wqq6z\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:10:21.127723 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.127686    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63086064-884c-4f33-bf02-298f0bf1d77e-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:10:21.127723 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.127730    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63086064-884c-4f33-bf02-298f0bf1d77e-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:10:21.195225 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.195194    2549 generic.go:356] "Generic (PLEG): container finished" podID="63086064-884c-4f33-bf02-298f0bf1d77e" containerID="c7706c55f1273898a6dedf74ab2eb0d49ec40bec0ad44874498e387d1e0479c0" exitCode=0
Sep 10 18:10:21.195362 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.195243    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"
Sep 10 18:10:21.195362 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.195278    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn" event={"ID":"63086064-884c-4f33-bf02-298f0bf1d77e","Type":"ContainerDied","Data":"c7706c55f1273898a6dedf74ab2eb0d49ec40bec0ad44874498e387d1e0479c0"}
Sep 10 18:10:21.195362 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.195328    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn" event={"ID":"63086064-884c-4f33-bf02-298f0bf1d77e","Type":"ContainerDied","Data":"d8b7e6a5435f95b897b15014f6d3a1ab5bee76031ecc1ecc2e3979fd7ba034da"}
Sep 10 18:10:21.195362 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.195350    2549 scope.go:117] "RemoveContainer" containerID="c7706c55f1273898a6dedf74ab2eb0d49ec40bec0ad44874498e387d1e0479c0"
Sep 10 18:10:21.205721 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.205687    2549 scope.go:117] "RemoveContainer" containerID="c7706c55f1273898a6dedf74ab2eb0d49ec40bec0ad44874498e387d1e0479c0"
Sep 10 18:10:21.206037 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:10:21.206017    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7706c55f1273898a6dedf74ab2eb0d49ec40bec0ad44874498e387d1e0479c0\": container with ID starting with c7706c55f1273898a6dedf74ab2eb0d49ec40bec0ad44874498e387d1e0479c0 not found: ID does not exist" containerID="c7706c55f1273898a6dedf74ab2eb0d49ec40bec0ad44874498e387d1e0479c0"
Sep 10 18:10:21.206090 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.206047    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7706c55f1273898a6dedf74ab2eb0d49ec40bec0ad44874498e387d1e0479c0"} err="failed to get container status \"c7706c55f1273898a6dedf74ab2eb0d49ec40bec0ad44874498e387d1e0479c0\": rpc error: code = NotFound desc = could not find container \"c7706c55f1273898a6dedf74ab2eb0d49ec40bec0ad44874498e387d1e0479c0\": container with ID starting with c7706c55f1273898a6dedf74ab2eb0d49ec40bec0ad44874498e387d1e0479c0 not found: ID does not exist"
Sep 10 18:10:21.219194 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.219162    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"]
Sep 10 18:10:21.222169 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:21.222134    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-4d9405-7cb74489b8-gplqn"]
Sep 10 18:10:22.029273 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:22.029240    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63086064-884c-4f33-bf02-298f0bf1d77e" path="/var/lib/kubelet/pods/63086064-884c-4f33-bf02-298f0bf1d77e/volumes"
Sep 10 18:10:24.432581 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:24.432549    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:10:38.242193 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:38.242160    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"]
Sep 10 18:10:38.242606 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:38.242366    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm" podUID="a36e6fe7-bbf7-447c-b15f-f74d38c96988" containerName="maas-api" containerID="cri-o://bb7180a7102f3eb3a0b4490a627178f3f561a096ab0b1db5e6a9307ff904856d" gracePeriod=30
Sep 10 18:10:38.487172 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:38.487146    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 18:10:38.580764 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:38.580662    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a36e6fe7-bbf7-447c-b15f-f74d38c96988-maas-api-tls\") pod \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") "
Sep 10 18:10:38.580764 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:38.580744    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a36e6fe7-bbf7-447c-b15f-f74d38c96988-openshift-service-ca\") pod \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") "
Sep 10 18:10:38.581015 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:38.580861    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95ksb\" (UniqueName: \"kubernetes.io/projected/a36e6fe7-bbf7-447c-b15f-f74d38c96988-kube-api-access-95ksb\") pod \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") "
Sep 10 18:10:38.581015 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:38.580880    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a36e6fe7-bbf7-447c-b15f-f74d38c96988-metrics-certs\") pod \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\" (UID: \"a36e6fe7-bbf7-447c-b15f-f74d38c96988\") "
Sep 10 18:10:38.581147 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:38.581118    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a36e6fe7-bbf7-447c-b15f-f74d38c96988-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a36e6fe7-bbf7-447c-b15f-f74d38c96988" (UID: "a36e6fe7-bbf7-447c-b15f-f74d38c96988"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:10:38.581254 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:38.581236    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a36e6fe7-bbf7-447c-b15f-f74d38c96988-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:10:38.582938 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:38.582887    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a36e6fe7-bbf7-447c-b15f-f74d38c96988-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a36e6fe7-bbf7-447c-b15f-f74d38c96988" (UID: "a36e6fe7-bbf7-447c-b15f-f74d38c96988"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:10:38.583051 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:38.582925    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a36e6fe7-bbf7-447c-b15f-f74d38c96988-kube-api-access-95ksb" (OuterVolumeSpecName: "kube-api-access-95ksb") pod "a36e6fe7-bbf7-447c-b15f-f74d38c96988" (UID: "a36e6fe7-bbf7-447c-b15f-f74d38c96988"). InnerVolumeSpecName "kube-api-access-95ksb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:10:38.583051 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:38.582967    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a36e6fe7-bbf7-447c-b15f-f74d38c96988-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a36e6fe7-bbf7-447c-b15f-f74d38c96988" (UID: "a36e6fe7-bbf7-447c-b15f-f74d38c96988"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:10:38.681970 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:38.681936    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95ksb\" (UniqueName: \"kubernetes.io/projected/a36e6fe7-bbf7-447c-b15f-f74d38c96988-kube-api-access-95ksb\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:10:38.681970 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:38.681964    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a36e6fe7-bbf7-447c-b15f-f74d38c96988-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:10:38.681970 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:38.681973    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a36e6fe7-bbf7-447c-b15f-f74d38c96988-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:10:39.285850 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:39.285810    2549 generic.go:356] "Generic (PLEG): container finished" podID="a36e6fe7-bbf7-447c-b15f-f74d38c96988" containerID="bb7180a7102f3eb3a0b4490a627178f3f561a096ab0b1db5e6a9307ff904856d" exitCode=0
Sep 10 18:10:39.286286 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:39.285859    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"
Sep 10 18:10:39.286286 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:39.285867    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm" event={"ID":"a36e6fe7-bbf7-447c-b15f-f74d38c96988","Type":"ContainerDied","Data":"bb7180a7102f3eb3a0b4490a627178f3f561a096ab0b1db5e6a9307ff904856d"}
Sep 10 18:10:39.286286 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:39.285918    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm" event={"ID":"a36e6fe7-bbf7-447c-b15f-f74d38c96988","Type":"ContainerDied","Data":"ab375bd454cfe365496dd48ad057a172645145920b09cf1db6d2ff8943071710"}
Sep 10 18:10:39.286286 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:39.285948    2549 scope.go:117] "RemoveContainer" containerID="bb7180a7102f3eb3a0b4490a627178f3f561a096ab0b1db5e6a9307ff904856d"
Sep 10 18:10:39.295048 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:39.295029    2549 scope.go:117] "RemoveContainer" containerID="bb7180a7102f3eb3a0b4490a627178f3f561a096ab0b1db5e6a9307ff904856d"
Sep 10 18:10:39.295327 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:10:39.295308    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb7180a7102f3eb3a0b4490a627178f3f561a096ab0b1db5e6a9307ff904856d\": container with ID starting with bb7180a7102f3eb3a0b4490a627178f3f561a096ab0b1db5e6a9307ff904856d not found: ID does not exist" containerID="bb7180a7102f3eb3a0b4490a627178f3f561a096ab0b1db5e6a9307ff904856d"
Sep 10 18:10:39.295378 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:39.295335    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb7180a7102f3eb3a0b4490a627178f3f561a096ab0b1db5e6a9307ff904856d"} err="failed to get container status \"bb7180a7102f3eb3a0b4490a627178f3f561a096ab0b1db5e6a9307ff904856d\": rpc error: code = NotFound desc = could not find container \"bb7180a7102f3eb3a0b4490a627178f3f561a096ab0b1db5e6a9307ff904856d\": container with ID starting with bb7180a7102f3eb3a0b4490a627178f3f561a096ab0b1db5e6a9307ff904856d not found: ID does not exist"
Sep 10 18:10:39.306953 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:39.306928    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"]
Sep 10 18:10:39.311310 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:39.311288    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-6c9ffc-68fcfc96d7-bf9tm"]
Sep 10 18:10:40.029964 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:40.029931    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a36e6fe7-bbf7-447c-b15f-f74d38c96988" path="/var/lib/kubelet/pods/a36e6fe7-bbf7-447c-b15f-f74d38c96988/volumes"
Sep 10 18:10:41.609037 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:41.609003    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:10:56.212541 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.212507    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"]
Sep 10 18:10:56.213088 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.212734    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96" podUID="4652b83c-f542-4587-ad05-6dbe47fc4fb2" containerName="maas-api" containerID="cri-o://d18e3be67ecd18d5f0014e8e6ce95d3d83a7d299a4811fce33347d51b2693078" gracePeriod=30
Sep 10 18:10:56.364585 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.364549    2549 generic.go:356] "Generic (PLEG): container finished" podID="4652b83c-f542-4587-ad05-6dbe47fc4fb2" containerID="d18e3be67ecd18d5f0014e8e6ce95d3d83a7d299a4811fce33347d51b2693078" exitCode=0
Sep 10 18:10:56.364781 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.364603    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96" event={"ID":"4652b83c-f542-4587-ad05-6dbe47fc4fb2","Type":"ContainerDied","Data":"d18e3be67ecd18d5f0014e8e6ce95d3d83a7d299a4811fce33347d51b2693078"}
Sep 10 18:10:56.458611 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.458588    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 18:10:56.537048 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.536963    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4652b83c-f542-4587-ad05-6dbe47fc4fb2-maas-api-tls\") pod \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") "
Sep 10 18:10:56.537048 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.537039    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4652b83c-f542-4587-ad05-6dbe47fc4fb2-metrics-certs\") pod \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") "
Sep 10 18:10:56.537286 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.537061    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4652b83c-f542-4587-ad05-6dbe47fc4fb2-openshift-service-ca\") pod \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") "
Sep 10 18:10:56.537286 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.537100    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2w2s\" (UniqueName: \"kubernetes.io/projected/4652b83c-f542-4587-ad05-6dbe47fc4fb2-kube-api-access-x2w2s\") pod \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\" (UID: \"4652b83c-f542-4587-ad05-6dbe47fc4fb2\") "
Sep 10 18:10:56.537467 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.537441    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4652b83c-f542-4587-ad05-6dbe47fc4fb2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4652b83c-f542-4587-ad05-6dbe47fc4fb2" (UID: "4652b83c-f542-4587-ad05-6dbe47fc4fb2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:10:56.539080 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.539044    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4652b83c-f542-4587-ad05-6dbe47fc4fb2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4652b83c-f542-4587-ad05-6dbe47fc4fb2" (UID: "4652b83c-f542-4587-ad05-6dbe47fc4fb2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:10:56.539213 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.539199    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4652b83c-f542-4587-ad05-6dbe47fc4fb2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4652b83c-f542-4587-ad05-6dbe47fc4fb2" (UID: "4652b83c-f542-4587-ad05-6dbe47fc4fb2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:10:56.539287 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.539271    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4652b83c-f542-4587-ad05-6dbe47fc4fb2-kube-api-access-x2w2s" (OuterVolumeSpecName: "kube-api-access-x2w2s") pod "4652b83c-f542-4587-ad05-6dbe47fc4fb2" (UID: "4652b83c-f542-4587-ad05-6dbe47fc4fb2"). InnerVolumeSpecName "kube-api-access-x2w2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:10:56.638645 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.638608    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4652b83c-f542-4587-ad05-6dbe47fc4fb2-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:10:56.638645 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.638638    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4652b83c-f542-4587-ad05-6dbe47fc4fb2-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:10:56.638645 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.638650    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2w2s\" (UniqueName: \"kubernetes.io/projected/4652b83c-f542-4587-ad05-6dbe47fc4fb2-kube-api-access-x2w2s\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:10:56.638896 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:56.638659    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4652b83c-f542-4587-ad05-6dbe47fc4fb2-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:10:57.370170 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:57.370140    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"
Sep 10 18:10:57.370596 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:57.370131    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96" event={"ID":"4652b83c-f542-4587-ad05-6dbe47fc4fb2","Type":"ContainerDied","Data":"4712962f729a4b61775406389d2872dcd883c526c20cb22e7279f9f56bd5a82a"}
Sep 10 18:10:57.370596 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:57.370288    2549 scope.go:117] "RemoveContainer" containerID="d18e3be67ecd18d5f0014e8e6ce95d3d83a7d299a4811fce33347d51b2693078"
Sep 10 18:10:57.392583 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:57.392554    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"]
Sep 10 18:10:57.400108 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:57.400084    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fc23cf-8448f4cf46-8wm96"]
Sep 10 18:10:58.029427 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:58.029388    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4652b83c-f542-4587-ad05-6dbe47fc4fb2" path="/var/lib/kubelet/pods/4652b83c-f542-4587-ad05-6dbe47fc4fb2/volumes"
Sep 10 18:10:59.199206 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:10:59.199165    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:11:07.513137 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:11:07.513098    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:12:30.100125 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:12:30.100086    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:12:56.901213 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:12:56.901173    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:13:06.399185 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:13:06.399144    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:13:13.706820 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:13:13.706782    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:13:19.302601 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:13:19.302558    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:13:36.501705 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:13:36.501667    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:13:42.408157 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:13:42.408117    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:15:10.404304 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:15:10.404265    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:15:19.488172 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:15:19.488131    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:15:29.098499 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:15:29.098461    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:15:41.595052 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:15:41.594967    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:15:58.701744 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:15:58.701708    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:16:07.894111 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:07.894071    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:16:16.022710 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.022660    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"]
Sep 10 18:16:16.023245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.023047    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4327a21-6426-4ea9-9a66-1c2db2957810" containerName="maas-api"
Sep 10 18:16:16.023245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.023058    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4327a21-6426-4ea9-9a66-1c2db2957810" containerName="maas-api"
Sep 10 18:16:16.023245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.023068    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1865e7-7700-4e3c-9c0d-bd5334198502" containerName="maas-api"
Sep 10 18:16:16.023245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.023073    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1865e7-7700-4e3c-9c0d-bd5334198502" containerName="maas-api"
Sep 10 18:16:16.023245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.023094    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a36e6fe7-bbf7-447c-b15f-f74d38c96988" containerName="maas-api"
Sep 10 18:16:16.023245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.023099    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="a36e6fe7-bbf7-447c-b15f-f74d38c96988" containerName="maas-api"
Sep 10 18:16:16.023245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.023105    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4652b83c-f542-4587-ad05-6dbe47fc4fb2" containerName="maas-api"
Sep 10 18:16:16.023245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.023110    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="4652b83c-f542-4587-ad05-6dbe47fc4fb2" containerName="maas-api"
Sep 10 18:16:16.023245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.023121    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63086064-884c-4f33-bf02-298f0bf1d77e" containerName="maas-api"
Sep 10 18:16:16.023245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.023126    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="63086064-884c-4f33-bf02-298f0bf1d77e" containerName="maas-api"
Sep 10 18:16:16.023245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.023183    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="4652b83c-f542-4587-ad05-6dbe47fc4fb2" containerName="maas-api"
Sep 10 18:16:16.023245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.023194    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="a36e6fe7-bbf7-447c-b15f-f74d38c96988" containerName="maas-api"
Sep 10 18:16:16.023245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.023202    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="63086064-884c-4f33-bf02-298f0bf1d77e" containerName="maas-api"
Sep 10 18:16:16.023245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.023213    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4327a21-6426-4ea9-9a66-1c2db2957810" containerName="maas-api"
Sep 10 18:16:16.023245 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.023222    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1865e7-7700-4e3c-9c0d-bd5334198502" containerName="maas-api"
Sep 10 18:16:16.026470 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.026444    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:16.029070 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.029044    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-n56s4\""
Sep 10 18:16:16.036140 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.036117    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"]
Sep 10 18:16:16.093612 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.093581    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52212c5f-2554-4364-810f-6fb61104ae8d-maas-api-tls\") pod \"maas-api-698bfd7f85-7rrc5\" (UID: \"52212c5f-2554-4364-810f-6fb61104ae8d\") " pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:16.093810 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.093619    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvzrl\" (UniqueName: \"kubernetes.io/projected/52212c5f-2554-4364-810f-6fb61104ae8d-kube-api-access-pvzrl\") pod \"maas-api-698bfd7f85-7rrc5\" (UID: \"52212c5f-2554-4364-810f-6fb61104ae8d\") " pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:16.093810 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.093682    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52212c5f-2554-4364-810f-6fb61104ae8d-metrics-certs\") pod \"maas-api-698bfd7f85-7rrc5\" (UID: \"52212c5f-2554-4364-810f-6fb61104ae8d\") " pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:16.093810 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.093726    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52212c5f-2554-4364-810f-6fb61104ae8d-openshift-service-ca\") pod \"maas-api-698bfd7f85-7rrc5\" (UID: \"52212c5f-2554-4364-810f-6fb61104ae8d\") " pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:16.194934 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.194900    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52212c5f-2554-4364-810f-6fb61104ae8d-maas-api-tls\") pod \"maas-api-698bfd7f85-7rrc5\" (UID: \"52212c5f-2554-4364-810f-6fb61104ae8d\") " pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:16.194934 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.194945    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvzrl\" (UniqueName: \"kubernetes.io/projected/52212c5f-2554-4364-810f-6fb61104ae8d-kube-api-access-pvzrl\") pod \"maas-api-698bfd7f85-7rrc5\" (UID: \"52212c5f-2554-4364-810f-6fb61104ae8d\") " pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:16.195162 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.194992    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52212c5f-2554-4364-810f-6fb61104ae8d-metrics-certs\") pod \"maas-api-698bfd7f85-7rrc5\" (UID: \"52212c5f-2554-4364-810f-6fb61104ae8d\") " pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:16.195162 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.195029    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52212c5f-2554-4364-810f-6fb61104ae8d-openshift-service-ca\") pod \"maas-api-698bfd7f85-7rrc5\" (UID: \"52212c5f-2554-4364-810f-6fb61104ae8d\") " pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:16.195753 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.195721    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52212c5f-2554-4364-810f-6fb61104ae8d-openshift-service-ca\") pod \"maas-api-698bfd7f85-7rrc5\" (UID: \"52212c5f-2554-4364-810f-6fb61104ae8d\") " pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:16.197504 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.197479    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52212c5f-2554-4364-810f-6fb61104ae8d-metrics-certs\") pod \"maas-api-698bfd7f85-7rrc5\" (UID: \"52212c5f-2554-4364-810f-6fb61104ae8d\") " pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:16.197721 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.197479    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52212c5f-2554-4364-810f-6fb61104ae8d-maas-api-tls\") pod \"maas-api-698bfd7f85-7rrc5\" (UID: \"52212c5f-2554-4364-810f-6fb61104ae8d\") " pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:16.202235 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.202214    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvzrl\" (UniqueName: \"kubernetes.io/projected/52212c5f-2554-4364-810f-6fb61104ae8d-kube-api-access-pvzrl\") pod \"maas-api-698bfd7f85-7rrc5\" (UID: \"52212c5f-2554-4364-810f-6fb61104ae8d\") " pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:16.340113 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.340013    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:16.489013 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.488973    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:16:16.609687 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.609658    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"]
Sep 10 18:16:16.611652 ip-10-0-139-245 kubenswrapper[2549]: W0910 18:16:16.611624    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52212c5f_2554_4364_810f_6fb61104ae8d.slice/crio-d531406440618b8553111b5d0f45f9ea71e90acd440637830ede78c48e3cc652 WatchSource:0}: Error finding container d531406440618b8553111b5d0f45f9ea71e90acd440637830ede78c48e3cc652: Status 404 returned error can't find the container with id d531406440618b8553111b5d0f45f9ea71e90acd440637830ede78c48e3cc652
Sep 10 18:16:16.613214 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.613191    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 18:16:16.808101 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:16.808061    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5" event={"ID":"52212c5f-2554-4364-810f-6fb61104ae8d","Type":"ContainerStarted","Data":"d531406440618b8553111b5d0f45f9ea71e90acd440637830ede78c48e3cc652"}
Sep 10 18:16:17.813808 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:17.813772    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5" event={"ID":"52212c5f-2554-4364-810f-6fb61104ae8d","Type":"ContainerStarted","Data":"c9ecfbc3bcbeaaad4e86a05bdec12107b67b7ef06e55dad0a96bc08235a9f22c"}
Sep 10 18:16:17.814197 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:17.813893    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:17.831270 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:17.831224    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5" podStartSLOduration=1.425264389 podStartE2EDuration="1.83120781s" totalImagesPullingTime="405.943421ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:16:17.828881939 +0000 UTC m=+2102.319933749" watchObservedRunningTime="2026-09-10 18:16:17.83120781 +0000 UTC m=+2102.322259617"
Sep 10 18:16:23.693102 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:23.693061    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:16:23.824482 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:23.824450    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-698bfd7f85-7rrc5"
Sep 10 18:16:30.800012 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:30.799972    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:16:31.253180 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:31.253138    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"]
Sep 10 18:16:31.253468 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:31.253414    2549 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx" podUID="db4e1904-529b-4ff2-96ca-1346137636f8" containerName="maas-api" containerID="cri-o://2c9243522a8c893151e7fddf5ee14ce100790d4ad09bc404ecec538374cae968" gracePeriod=30
Sep 10 18:16:32.496478 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.496452    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 18:16:32.543669 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.543635    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96jv5\" (UniqueName: \"kubernetes.io/projected/db4e1904-529b-4ff2-96ca-1346137636f8-kube-api-access-96jv5\") pod \"db4e1904-529b-4ff2-96ca-1346137636f8\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") "
Sep 10 18:16:32.543872 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.543717    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-maas-api-tls\") pod \"db4e1904-529b-4ff2-96ca-1346137636f8\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") "
Sep 10 18:16:32.543872 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.543766    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db4e1904-529b-4ff2-96ca-1346137636f8-openshift-service-ca\") pod \"db4e1904-529b-4ff2-96ca-1346137636f8\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") "
Sep 10 18:16:32.543872 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.543802    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-metrics-certs\") pod \"db4e1904-529b-4ff2-96ca-1346137636f8\" (UID: \"db4e1904-529b-4ff2-96ca-1346137636f8\") "
Sep 10 18:16:32.544177 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.544146    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db4e1904-529b-4ff2-96ca-1346137636f8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "db4e1904-529b-4ff2-96ca-1346137636f8" (UID: "db4e1904-529b-4ff2-96ca-1346137636f8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:16:32.545965 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.545936    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "db4e1904-529b-4ff2-96ca-1346137636f8" (UID: "db4e1904-529b-4ff2-96ca-1346137636f8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:16:32.546067 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.546004    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db4e1904-529b-4ff2-96ca-1346137636f8-kube-api-access-96jv5" (OuterVolumeSpecName: "kube-api-access-96jv5") pod "db4e1904-529b-4ff2-96ca-1346137636f8" (UID: "db4e1904-529b-4ff2-96ca-1346137636f8"). InnerVolumeSpecName "kube-api-access-96jv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:16:32.546347 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.546327    2549 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "db4e1904-529b-4ff2-96ca-1346137636f8" (UID: "db4e1904-529b-4ff2-96ca-1346137636f8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:16:32.645330 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.645235    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96jv5\" (UniqueName: \"kubernetes.io/projected/db4e1904-529b-4ff2-96ca-1346137636f8-kube-api-access-96jv5\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:16:32.645330 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.645272    2549 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-maas-api-tls\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:16:32.645330 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.645282    2549 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db4e1904-529b-4ff2-96ca-1346137636f8-openshift-service-ca\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:16:32.645330 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.645292    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db4e1904-529b-4ff2-96ca-1346137636f8-metrics-certs\") on node \"ip-10-0-139-245.ec2.internal\" DevicePath \"\""
Sep 10 18:16:32.881586 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.881550    2549 generic.go:356] "Generic (PLEG): container finished" podID="db4e1904-529b-4ff2-96ca-1346137636f8" containerID="2c9243522a8c893151e7fddf5ee14ce100790d4ad09bc404ecec538374cae968" exitCode=0
Sep 10 18:16:32.881809 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.881608    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx" event={"ID":"db4e1904-529b-4ff2-96ca-1346137636f8","Type":"ContainerDied","Data":"2c9243522a8c893151e7fddf5ee14ce100790d4ad09bc404ecec538374cae968"}
Sep 10 18:16:32.881809 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.881617    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"
Sep 10 18:16:32.881809 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.881674    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54b46465b9-htdqx" event={"ID":"db4e1904-529b-4ff2-96ca-1346137636f8","Type":"ContainerDied","Data":"7482ca272f96c199722cefe5a69e6e451ab822ff31bde9d25cce2a8b3d6c2f8f"}
Sep 10 18:16:32.881809 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.881725    2549 scope.go:117] "RemoveContainer" containerID="2c9243522a8c893151e7fddf5ee14ce100790d4ad09bc404ecec538374cae968"
Sep 10 18:16:32.891326 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.891308    2549 scope.go:117] "RemoveContainer" containerID="2c9243522a8c893151e7fddf5ee14ce100790d4ad09bc404ecec538374cae968"
Sep 10 18:16:32.891593 ip-10-0-139-245 kubenswrapper[2549]: E0910 18:16:32.891577    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c9243522a8c893151e7fddf5ee14ce100790d4ad09bc404ecec538374cae968\": container with ID starting with 2c9243522a8c893151e7fddf5ee14ce100790d4ad09bc404ecec538374cae968 not found: ID does not exist" containerID="2c9243522a8c893151e7fddf5ee14ce100790d4ad09bc404ecec538374cae968"
Sep 10 18:16:32.891644 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.891600    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c9243522a8c893151e7fddf5ee14ce100790d4ad09bc404ecec538374cae968"} err="failed to get container status \"2c9243522a8c893151e7fddf5ee14ce100790d4ad09bc404ecec538374cae968\": rpc error: code = NotFound desc = could not find container \"2c9243522a8c893151e7fddf5ee14ce100790d4ad09bc404ecec538374cae968\": container with ID starting with 2c9243522a8c893151e7fddf5ee14ce100790d4ad09bc404ecec538374cae968 not found: ID does not exist"
Sep 10 18:16:32.903142 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.902983    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"]
Sep 10 18:16:32.904591 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:32.904571    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-54b46465b9-htdqx"]
Sep 10 18:16:34.028699 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:34.028652    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db4e1904-529b-4ff2-96ca-1346137636f8" path="/var/lib/kubelet/pods/db4e1904-529b-4ff2-96ca-1346137636f8/volumes"
Sep 10 18:16:39.092237 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:39.092202    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:16:46.893752 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:46.893715    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ld75g"]
Sep 10 18:16:50.081629 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:50.081598    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f7c475b8-9vwpt_2be677c4-2c33-4fa0-b1a3-978dc1582f63/authorino/0.log"
Sep 10 18:16:55.960622 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:55.960586    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556447cf57-n8n46_7f310d90-52be-4156-8478-084ecd2f768d/copy-manifests/0.log"
Sep 10 18:16:55.966370 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:55.966341    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556447cf57-n8n46_7f310d90-52be-4156-8478-084ecd2f768d/manager/0.log"
Sep 10 18:16:56.431870 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:56.431787    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-698bfd7f85-7rrc5_52212c5f-2554-4364-810f-6fb61104ae8d/maas-api/0.log"
Sep 10 18:16:57.660646 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:57.660618    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f7c475b8-9vwpt_2be677c4-2c33-4fa0-b1a3-978dc1582f63/authorino/0.log"
Sep 10 18:16:58.244918 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:58.244883    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-ld75g_12cca286-9d95-486f-9f8e-b0f4e530cb24/limitador/0.log"
Sep 10 18:16:59.081879 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:16:59.081846    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c775fbbf-z4v6q_b982c773-c239-4c45-be2c-6614850d2215/kube-auth-proxy/0.log"
Sep 10 18:17:00.366594 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:00.366544    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fc7bk4_827c50a7-3b56-481e-8a22-6ae4cf6aaf5c/main/0.log"
Sep 10 18:17:00.584254 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:00.584219    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-9g6b9_4a54e3a8-887c-4f2c-9efe-6145059199f1/main/0.log"
Sep 10 18:17:09.392644 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:09.392548    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mj85b_8cae49c8-331a-409b-8f6a-7d096c469ca5/global-pull-secret-syncer/0.log"
Sep 10 18:17:09.534336 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:09.534297    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zbhnk_81cddb05-3fd0-4ed3-83b6-e8d96dd71dcd/konnectivity-agent/0.log"
Sep 10 18:17:09.599706 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:09.599667    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-245.ec2.internal_71f8e2acfe6edbc5bafb0547bd10904c/haproxy/0.log"
Sep 10 18:17:14.828833 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:14.828798    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f7c475b8-9vwpt_2be677c4-2c33-4fa0-b1a3-978dc1582f63/authorino/0.log"
Sep 10 18:17:14.982003 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:14.981968    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-ld75g_12cca286-9d95-486f-9f8e-b0f4e530cb24/limitador/0.log"
Sep 10 18:17:16.570685 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:16.570575    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-8468bdbf64-f754m_f637a746-636f-404c-9593-3e8b393b1663/metrics-server/0.log"
Sep 10 18:17:16.618355 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:16.618321    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dbzh4_baf11260-22e8-4976-91f0-c417469342da/node-exporter/0.log"
Sep 10 18:17:16.636762 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:16.636732    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dbzh4_baf11260-22e8-4976-91f0-c417469342da/kube-rbac-proxy/0.log"
Sep 10 18:17:16.653015 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:16.652991    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dbzh4_baf11260-22e8-4976-91f0-c417469342da/init-textfile/0.log"
Sep 10 18:17:16.923013 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:16.922965    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2bcb7a83-181a-4c86-8bb4-313deb152061/prometheus/0.log"
Sep 10 18:17:16.943959 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:16.943932    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2bcb7a83-181a-4c86-8bb4-313deb152061/config-reloader/0.log"
Sep 10 18:17:16.965367 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:16.965332    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2bcb7a83-181a-4c86-8bb4-313deb152061/thanos-sidecar/0.log"
Sep 10 18:17:16.983750 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:16.983716    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2bcb7a83-181a-4c86-8bb4-313deb152061/kube-rbac-proxy-web/0.log"
Sep 10 18:17:17.005335 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:17.005312    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2bcb7a83-181a-4c86-8bb4-313deb152061/kube-rbac-proxy/0.log"
Sep 10 18:17:17.034459 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:17.034428    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2bcb7a83-181a-4c86-8bb4-313deb152061/kube-rbac-proxy-thanos/0.log"
Sep 10 18:17:17.051902 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:17.051871    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2bcb7a83-181a-4c86-8bb4-313deb152061/init-config-reloader/0.log"
Sep 10 18:17:17.079622 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:17.079587    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-gcdnv_b77d1245-6e62-4c4d-afdf-471974a7fa89/prometheus-operator/0.log"
Sep 10 18:17:17.092034 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:17.091989    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-gcdnv_b77d1245-6e62-4c4d-afdf-471974a7fa89/kube-rbac-proxy/0.log"
Sep 10 18:17:17.126398 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:17.126321    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-jrgfx_65998d10-dfba-422e-bf2e-692ca983f800/prometheus-operator-admission-webhook/0.log"
Sep 10 18:17:17.251081 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:17.251054    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dd6ffb6df-m97z6_3901e250-2092-4cde-9ef7-d3602bde097e/thanos-query/0.log"
Sep 10 18:17:17.268614 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:17.268587    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dd6ffb6df-m97z6_3901e250-2092-4cde-9ef7-d3602bde097e/kube-rbac-proxy-web/0.log"
Sep 10 18:17:17.287637 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:17.287613    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dd6ffb6df-m97z6_3901e250-2092-4cde-9ef7-d3602bde097e/kube-rbac-proxy/0.log"
Sep 10 18:17:17.305641 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:17.305616    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dd6ffb6df-m97z6_3901e250-2092-4cde-9ef7-d3602bde097e/prom-label-proxy/0.log"
Sep 10 18:17:17.321879 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:17.321844    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dd6ffb6df-m97z6_3901e250-2092-4cde-9ef7-d3602bde097e/kube-rbac-proxy-rules/0.log"
Sep 10 18:17:17.343564 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:17.343539    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dd6ffb6df-m97z6_3901e250-2092-4cde-9ef7-d3602bde097e/kube-rbac-proxy-metrics/0.log"
Sep 10 18:17:19.921246 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:19.921213    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-4nnsl_b459cafb-dae0-4cfd-9859-ed15e033dc5c/download-server/0.log"
Sep 10 18:17:20.132332 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.132298    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"]
Sep 10 18:17:20.132668 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.132656    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db4e1904-529b-4ff2-96ca-1346137636f8" containerName="maas-api"
Sep 10 18:17:20.132736 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.132670    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="db4e1904-529b-4ff2-96ca-1346137636f8" containerName="maas-api"
Sep 10 18:17:20.132776 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.132747    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="db4e1904-529b-4ff2-96ca-1346137636f8" containerName="maas-api"
Sep 10 18:17:20.137431 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.137406    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.139980 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.139952    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h9jz8\"/\"openshift-service-ca.crt\""
Sep 10 18:17:20.139980 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.139969    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-h9jz8\"/\"default-dockercfg-zgjz4\""
Sep 10 18:17:20.140357 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.139991    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h9jz8\"/\"kube-root-ca.crt\""
Sep 10 18:17:20.143819 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.143795    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"]
Sep 10 18:17:20.268001 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.267973    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eb73db00-d69a-43ac-90c2-bb07de3b42e4-podres\") pod \"perf-node-gather-daemonset-2xztr\" (UID: \"eb73db00-d69a-43ac-90c2-bb07de3b42e4\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.268180 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.268010    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eb73db00-d69a-43ac-90c2-bb07de3b42e4-proc\") pod \"perf-node-gather-daemonset-2xztr\" (UID: \"eb73db00-d69a-43ac-90c2-bb07de3b42e4\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.268180 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.268047    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xglfn\" (UniqueName: \"kubernetes.io/projected/eb73db00-d69a-43ac-90c2-bb07de3b42e4-kube-api-access-xglfn\") pod \"perf-node-gather-daemonset-2xztr\" (UID: \"eb73db00-d69a-43ac-90c2-bb07de3b42e4\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.268180 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.268084    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb73db00-d69a-43ac-90c2-bb07de3b42e4-lib-modules\") pod \"perf-node-gather-daemonset-2xztr\" (UID: \"eb73db00-d69a-43ac-90c2-bb07de3b42e4\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.268180 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.268120    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb73db00-d69a-43ac-90c2-bb07de3b42e4-sys\") pod \"perf-node-gather-daemonset-2xztr\" (UID: \"eb73db00-d69a-43ac-90c2-bb07de3b42e4\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.368759 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.368720    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb73db00-d69a-43ac-90c2-bb07de3b42e4-lib-modules\") pod \"perf-node-gather-daemonset-2xztr\" (UID: \"eb73db00-d69a-43ac-90c2-bb07de3b42e4\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.368958 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.368771    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb73db00-d69a-43ac-90c2-bb07de3b42e4-sys\") pod \"perf-node-gather-daemonset-2xztr\" (UID: \"eb73db00-d69a-43ac-90c2-bb07de3b42e4\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.368958 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.368835    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eb73db00-d69a-43ac-90c2-bb07de3b42e4-podres\") pod \"perf-node-gather-daemonset-2xztr\" (UID: \"eb73db00-d69a-43ac-90c2-bb07de3b42e4\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.368958 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.368853    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eb73db00-d69a-43ac-90c2-bb07de3b42e4-proc\") pod \"perf-node-gather-daemonset-2xztr\" (UID: \"eb73db00-d69a-43ac-90c2-bb07de3b42e4\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.368958 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.368877    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xglfn\" (UniqueName: \"kubernetes.io/projected/eb73db00-d69a-43ac-90c2-bb07de3b42e4-kube-api-access-xglfn\") pod \"perf-node-gather-daemonset-2xztr\" (UID: \"eb73db00-d69a-43ac-90c2-bb07de3b42e4\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.368958 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.368899    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eb73db00-d69a-43ac-90c2-bb07de3b42e4-lib-modules\") pod \"perf-node-gather-daemonset-2xztr\" (UID: \"eb73db00-d69a-43ac-90c2-bb07de3b42e4\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.368958 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.368921    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eb73db00-d69a-43ac-90c2-bb07de3b42e4-sys\") pod \"perf-node-gather-daemonset-2xztr\" (UID: \"eb73db00-d69a-43ac-90c2-bb07de3b42e4\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.368958 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.368949    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eb73db00-d69a-43ac-90c2-bb07de3b42e4-proc\") pod \"perf-node-gather-daemonset-2xztr\" (UID: \"eb73db00-d69a-43ac-90c2-bb07de3b42e4\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.369235 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.368985    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eb73db00-d69a-43ac-90c2-bb07de3b42e4-podres\") pod \"perf-node-gather-daemonset-2xztr\" (UID: \"eb73db00-d69a-43ac-90c2-bb07de3b42e4\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.376948 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.376924    2549 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xglfn\" (UniqueName: \"kubernetes.io/projected/eb73db00-d69a-43ac-90c2-bb07de3b42e4-kube-api-access-xglfn\") pod \"perf-node-gather-daemonset-2xztr\" (UID: \"eb73db00-d69a-43ac-90c2-bb07de3b42e4\") " pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.449238 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.449197    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:20.716985 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:20.716958    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"]
Sep 10 18:17:20.718914 ip-10-0-139-245 kubenswrapper[2549]: W0910 18:17:20.718885    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podeb73db00_d69a_43ac_90c2_bb07de3b42e4.slice/crio-fa30d1e32853c261b49290cbd45c2acce2134d9398dd94061fb6b71c842fa0eb WatchSource:0}: Error finding container fa30d1e32853c261b49290cbd45c2acce2134d9398dd94061fb6b71c842fa0eb: Status 404 returned error can't find the container with id fa30d1e32853c261b49290cbd45c2acce2134d9398dd94061fb6b71c842fa0eb
Sep 10 18:17:21.096182 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:21.096091    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr" event={"ID":"eb73db00-d69a-43ac-90c2-bb07de3b42e4","Type":"ContainerStarted","Data":"3368649508f4fd28463e4f72e47704f5652036f849805a04a72461e3189d81b2"}
Sep 10 18:17:21.096182 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:21.096140    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr" event={"ID":"eb73db00-d69a-43ac-90c2-bb07de3b42e4","Type":"ContainerStarted","Data":"fa30d1e32853c261b49290cbd45c2acce2134d9398dd94061fb6b71c842fa0eb"}
Sep 10 18:17:21.096575 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:21.096190    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:21.111313 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:21.111246    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr" podStartSLOduration=1.111229268 podStartE2EDuration="1.111229268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:17:21.110370577 +0000 UTC m=+2165.601422379" watchObservedRunningTime="2026-09-10 18:17:21.111229268 +0000 UTC m=+2165.602281069"
Sep 10 18:17:21.851168 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:21.851135    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kgrgd_65e67805-ef05-4076-9cdc-71ece6139849/dns/0.log"
Sep 10 18:17:21.869366 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:21.869332    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kgrgd_65e67805-ef05-4076-9cdc-71ece6139849/kube-rbac-proxy/0.log"
Sep 10 18:17:21.964234 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:21.964207    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kdng8_df747e4e-bb42-420a-90e5-318f0b4f31e8/dns-node-resolver/0.log"
Sep 10 18:17:22.466956 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:22.466919    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-58d4769c69-rjtx9_fff37c0e-0b99-46d0-afc5-a6e2132641f6/registry/0.log"
Sep 10 18:17:22.510752 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:22.510719    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jpdgj_1c14a9e5-fec4-4b25-b3a2-99d9bb6be063/node-ca/0.log"
Sep 10 18:17:23.660974 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:23.660938    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c775fbbf-z4v6q_b982c773-c239-4c45-be2c-6614850d2215/kube-auth-proxy/0.log"
Sep 10 18:17:24.636375 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:24.636347    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9gqnm_36e2db1a-137d-45af-8cdd-3d2372c27efb/serve-healthcheck-canary/0.log"
Sep 10 18:17:25.245139 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:25.245102    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v92d8_defc63f5-ba34-4dd3-bb22-fd991d6be1ac/kube-rbac-proxy/0.log"
Sep 10 18:17:25.261583 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:25.261554    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v92d8_defc63f5-ba34-4dd3-bb22-fd991d6be1ac/exporter/0.log"
Sep 10 18:17:25.280519 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:25.280485    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v92d8_defc63f5-ba34-4dd3-bb22-fd991d6be1ac/extractor/0.log"
Sep 10 18:17:27.112400 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:27.112372    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-h9jz8/perf-node-gather-daemonset-2xztr"
Sep 10 18:17:27.811252 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:27.811214    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556447cf57-n8n46_7f310d90-52be-4156-8478-084ecd2f768d/manager/0.log"
Sep 10 18:17:27.826587 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:27.826562    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556447cf57-n8n46_7f310d90-52be-4156-8478-084ecd2f768d/copy-manifests/0.log"
Sep 10 18:17:29.112846 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:29.112816    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-746f98bbcc-c276f_086300ad-3bb7-4aaa-b982-ab577f740ba8/manager/0.log"
Sep 10 18:17:31.226172 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:31.226141    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c775fbbf-z4v6q_b982c773-c239-4c45-be2c-6614850d2215/kube-auth-proxy/0.log"
Sep 10 18:17:33.274040 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:33.274008    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c775fbbf-z4v6q_b982c773-c239-4c45-be2c-6614850d2215/kube-auth-proxy/0.log"
Sep 10 18:17:34.494553 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:34.494504    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-7hrx5_c1c55e61-318c-4184-a18c-f9505f66f332/migrator/0.log"
Sep 10 18:17:34.510741 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:34.510714    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-7hrx5_c1c55e61-318c-4184-a18c-f9505f66f332/graceful-termination/0.log"
Sep 10 18:17:35.964856 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:35.964824    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9lfxv_213a7f60-9ca9-48fc-824e-6c5c14451aad/kube-multus-additional-cni-plugins/0.log"
Sep 10 18:17:35.981203 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:35.981167    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9lfxv_213a7f60-9ca9-48fc-824e-6c5c14451aad/egress-router-binary-copy/0.log"
Sep 10 18:17:36.000925 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:36.000901    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9lfxv_213a7f60-9ca9-48fc-824e-6c5c14451aad/cni-plugins/0.log"
Sep 10 18:17:36.019064 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:36.019037    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9lfxv_213a7f60-9ca9-48fc-824e-6c5c14451aad/bond-cni-plugin/0.log"
Sep 10 18:17:36.035434 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:36.035405    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9lfxv_213a7f60-9ca9-48fc-824e-6c5c14451aad/routeoverride-cni/0.log"
Sep 10 18:17:36.051315 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:36.051289    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9lfxv_213a7f60-9ca9-48fc-824e-6c5c14451aad/whereabouts-cni-bincopy/0.log"
Sep 10 18:17:36.067134 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:36.067108    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9lfxv_213a7f60-9ca9-48fc-824e-6c5c14451aad/whereabouts-cni/0.log"
Sep 10 18:17:36.277025 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:36.276940    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-r676t_d3bb440a-577b-4df2-bb07-dd8ce1870717/kube-multus/0.log"
Sep 10 18:17:36.411254 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:36.411211    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tnrvx_78bdfa50-029c-4951-9ca4-4bfa31adf9a8/network-metrics-daemon/0.log"
Sep 10 18:17:36.426310 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:36.426273    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tnrvx_78bdfa50-029c-4951-9ca4-4bfa31adf9a8/kube-rbac-proxy/0.log"
Sep 10 18:17:37.610117 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:37.610081    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krlkl_50a3c219-25ae-42e8-9ed7-ae827178226a/ovn-controller/0.log"
Sep 10 18:17:37.623193 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:37.623164    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krlkl_50a3c219-25ae-42e8-9ed7-ae827178226a/ovn-acl-logging/0.log"
Sep 10 18:17:37.641569 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:37.641526    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krlkl_50a3c219-25ae-42e8-9ed7-ae827178226a/kube-rbac-proxy-node/0.log"
Sep 10 18:17:37.659929 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:37.659894    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krlkl_50a3c219-25ae-42e8-9ed7-ae827178226a/kube-rbac-proxy-ovn-metrics/0.log"
Sep 10 18:17:37.676737 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:37.676684    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krlkl_50a3c219-25ae-42e8-9ed7-ae827178226a/northd/0.log"
Sep 10 18:17:37.697759 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:37.697737    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krlkl_50a3c219-25ae-42e8-9ed7-ae827178226a/nbdb/0.log"
Sep 10 18:17:37.714976 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:37.714946    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krlkl_50a3c219-25ae-42e8-9ed7-ae827178226a/sbdb/0.log"
Sep 10 18:17:37.794311 ip-10-0-139-245 kubenswrapper[2549]: I0910 18:17:37.794282    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-krlkl_50a3c219-25ae-42e8-9ed7-ae827178226a/ovnkube-controller/0.log"